Computational Art

Team: Drew Gleeman, Cory Levinson, Shaagnik Mukherji, Anton Pugh

Concept:
Rotary Phone:
Today, hardly a minute goes by without someone “tweeting” what they’re doing.  The rotary phone in the SmartArmoire brings that back to older days before full keyboards, then takes that back further by replacing the keypad with a rotary dial, bringing a magical feel to a vintage machine.

Music:
Like the rest of the ‘apps’ within the SmartArmoire, the music app serves the purpose of bringing modern day technological accessibility of an iPod through the antiquated feel of physical vinyl records and record sleeves.  The music app integrates vinyl records (both new and old) with a computational waveform/metadata display, along with track skip and shuffling abilities, to bring a vintage feel to our modern day interactions with music playback devices.

Maps:
Without a doubt, one of the single greatest achievements of modern technology is the invention of GPS. It is the product of advances from a wide array of fields, from computer science to aerospace engineering to theoretical physics. In just a few short years, it has transformed the notion of getting lost from a constant hazard of travelling into something that is virtually impossible. Children of the twenty-first century will have no ideas about the difficulties involved in reading and navigating with a map, and this facet of the project aims to incorporate these challenges into the modern idea of a GPS.

Video Games:
The overwhelming popularity of the game Angry Birds is uncontested. One of the primary reasons the game is so popular is the control scheme which is both easy and intuitive on mobile devices. So it was an interesting challenge to bring this game back into the days of analog technology. We attempted to make the controller look very similar to the slingshot in the game, as well as make the controls as analogous as possible to those in the game.

Process:
Rotary Phone:
The rotary phone’s signal is interrupted by the dial passing numbers.  This signal is used to identify what numbers are being dialed and a message is pulled from them.  A photoresistor determines whether the phone is on the hook or not, and when the phone is hung up, the message is sent.  When a tweet is received that is addressed @SmartArmoire or with the hashtag #SmartArmoire, a bell is rung to simulate the phone ringing.  Upon answering the phone, a computer voice reads the tweet out.  Additionally, the phone displays TwitterVision, an earlier project, when hung up.

Music:
Each of the vinyl record sleeves has a QR-Code fixed to the back of the record.  When the vinyl cover is inserted into the slot in the armoire, a flex sensor recognizes the record sleeve, and a webcam begins reading the QR=Code from the sleeve.  Once the QR-Code is recognized, the album begins playing, a display is shown on the television, and the track skip, play/pause, and shuffle buttons are enabled.  Upon removing the record, the music stops, until a new record is inserted in its place.

Video Games:
Initially we used an NES controller for controlling Angry Birds, but we wanted to take it a step further back in time. NES controllers are digital, and we wanted to use analog technology. So, we developed a slingshot controller that functions essentially the same as a normal slingshot, but its motions are transferred into the game.

Technical Description:
Music:
The music app uses QR codes that are taped to the bottom right of the back of each record sleeve.  When the record is inserted into the slot in the SmartArmoire, a flex sensor triggers a camera in the drawer to begin reading the QR code.  Once the code is recognized, the album begins playback.  All mp3’s from the 12 records available are stored locally with appropriate metadata (including album art) on the computer running processing, and playback is controlled via the minim processing library.  The album art along with tracklistings are displayed across the TV when the music app is selected with the VGA switcher, and the physical buttons control playback via an arduino.

Maps:
This portion of the project uses a camera hidden inside a lamp above the map. The camera captures images of the map and sends the information to the processing app. The app uses the OpenCV library and blob detection to identify the corners of the map as well as the start and end locations. It then converts these locations into latitude and longitude coordinates, and sends a query to the google maps web service. It receives a response including the directions in JSON format which it then prints to the screen.

Rotary Phone:
When the phone is hung up, it displays tweets from a certain location in the country determined by the direction and position of the antenna and dial on top of the TV cabinet. The program calculates this position by reading the values on the attached potentiometers and converting the readings into GPS coordinates. The app then filters a stream of tweets for ones sent from that specific geolocation and displays them on the screen. The app performs a similar function in order to detect tweets with the hash tag #SmartArmoire. If it detects a tweet with that tag, it causes the bell to ring and reads the tweet aloud when the phone is picked up. When the phone is off the hook and a user is dialing, the arduino reads the pulse signals sent by the phone in order to determine which number was dialed. The app translates this into letters.

Instructions:
Rotary Phone:
Lift the handset and dial your message, then hang up to send!  If the phone rings, make sure to answer it!

Music:
Place a record inside the sleeve and use the buttons to select a track, pause playback, or shuffle through a album!

Maps:
Place the pawn on the desired starting location, and the crosshair on the destination. Press the green button and the TV will display turn-by-turn directions, distance to destination, and estimated time.

Video Games:
To move the cursor around and to press down (for both grabbing the bird and clicking on things) use the analog joystick at the center of the slingshot. Once the bird is grabbed pull back on the slingshot (slowly works best) and release to let the bird fly.

Images:

Team: Drew Pompa

Background

Originally, I set out to create a type of “digital pet.” While this idea brings to mind images of Tamagotchi and Pokemon, I was aiming more for the real. The plan was to project a digital pet into a very realistic pet environment- that is, an aquarium, fully equipped with wood shavings and other details to complete the scene. For interaction, I had a stereo mic set up against the front wall of the aquarium (though hidden), which could detect when someone was tapping on the glass and come up with an idea of where the tap occurred, triggering action in the pet. The system worked pretty well when I was testing it (in a quiet room), but I noticed it was very sensitive to noise, which I assumed Tishman Hall would contain lots of. Despite my efforts at thresholding, I couldn’t seem to produce a “tight” response from the pet. Therefore, I had to change how the viewer interacts with the piece. I updated the concept, adding a mouse-based level of interactivity. However, this seemed to destroy the “analog” feel of the project. I finally decided to go back to my original, sound-based design (although slightly altered and with less equipment), and one all-nighter later, it was complete.

Concept

In this project, I aimed to explore the boundaries between the digital and the real through the notion of pets. Presented with a box containing a “pet,” how would users interact with it? Would their decisions be based on experience with real pets? I wanted to find out, and challenge people to interact with a small and simple piece.

Technical Description

The major technical component of the project deals with sound. Taps are detected using audio thresholding. In fact, three thresholds are present, and exceeding each one contributes to a different emotional effect on the pet. A slight notion of direction of taps was also included, but only if there was a significant discrepancy between the left and right channels of audio. However, this usually didn’t happen, because the mics are so close to each other, and they usually recorded about the same level from both channels. When activated, the pet will favor that side (left or right) of the screen when exploring the source of the tap.

The rest of the code works to animate and update the pet. Routine procedures include moving it to a new location, changing its color/expression, causing it to blink, and other actions. While none of these sorts of methods were very difficult to work out, it has been a challenge to call them sparingly enough to keep the program running smoothly while still presenting the “realistic” actions of a creature.

The hardware involved with this project is quite conventional. The “black box” itself is a cardboard box with black felt glued onto it. Inside is a computer monitor, which receives the video from the Processing code on my laptop via VGA. All sound is recorded using the laptop’s internal stereo mic, since the laptop is also inside of the box.

Results

I’m very happy with the project, and I think viewers also enjoyed it. It was very simple and easy to use, yet autonomous, since it essentially challenged you to interact with it. I’m especially pleased with the overall look of the project. By stripping unnecessary details away, and reducing the visuals to the pixel-level, I feel like I’ve actually strengthened the aesthetic considerably. By framing it in a simple black box and presenting it in a “no-nonsense” manner on a table, the simplicity of the piece was emphasized. I feel like the linked video below displays these results quite well.

Code/Demonstration Video

The Processing sketch and executables can be found here. A demonstration video can be found here.

Team:Miguel Abreu, Fan Guo

Concept:
In this project we reflect on the propagation of information. We realized that with the prevalence of network and digital devices, the actions of sending or receiving messages has become increasingly trivial. Yet, since the transmission process is almost instantaneous, all the nuances attached to communication through digital devices are lost. We wanted to create an experience where the emotional and poetic aspects of sending/delivering/receiving messages are emphasized. The theme we decided would best emphasize the delivery of a message was wind.

Process:
The communication devices we decided to use in this project were two laptops. Our goal was to simulate a message system experience. Communication from one person to another, through the use of the laptops, would be enhanced due to a graphical display that allows you to see your message being sent. This graphical display would also signal to the recipient that he/she’s receiving a message which would create a sense of anticipation.

Communication between the laptops was done using the processing network library. Each laptop ran a program that served as the message box system that used the University of Michigan’s wireless network. This program allowed a person to send and receive messages through the creation of a server. In order to show the transmission of a message in process (and to send messages), we created a separate program that acted as the bridge between the computers. Since each laptop would be running a server for itself, this bridge program created two client variables that connected to each server and received/sent any messages as needed.

Our next hurdle became the visual representation of the message. We first decided on the use of a particle system to represent each transmission. In order to send strings of text through a particle system, each character would need its own particle. Each particle would then be seen on the screen as its respective character and behave according to whatever  movement system we chose. Since we decided our theme was going to be wind, we used a number of random variable selection and physics calculations to best represent a wind-like behavior for each particle. Since each message was going from one computer to another, we created a funnel point for all the particles to get sucked into. This funnel point was where the receiving laptop would be during our presentation. A similar animation was done to the laptop server program where your message would get sucked into the middle of the screen.

Lastly we needed a mechanism for sending the message. We wanted to make an intuitive device that also only coincided with our wind theme. The object we used was a pinwheel.  In order to send a message the person would blow into the pinwheel. We used a microphone in the center of the pinwheel to detect when a message is needed to be sent instead of detecting the motion of the pinwheel. This microphone was then hooked into an arduino board which was then hooked up to its respective laptop. The laptops server program received the signal from the arduino board to send the message and then the transmission process would begin.

Result:
We were pleased with the elegance of the characters behaved when being sent as well as their visual effects. Sent and received messages through the campus network had little to no issue. The arduino boards also worked as designed. Sending messages became a fun experience during the exhibition as well. We not only successfully create the sense of anticipation of a message but gave message transmission an elegant display.

Lost in Transmission

Team: Jesse Hendrickson, Ross Henri, Becky Moore, Sara Frederixon

Fireworks is a unique, exciting experience where the user get to design and launch their own firework.  The experience is all about their interaction from drawing out their own image to using their whole body to launch their creation into the virtual night sky.  The user starts by operating our own firework creation tool to draw out any shape or image they can imagine.   They then enter our tent to be immersed into an astounding 3D event.  The image they have created is waiting in their hand to be sent flying; all they need to do is cast out their arm.  In front of them is now a beautiful explosion of particles and color in the exact form of their custom shape.   It is an experience all of their own just for them.

The process starts with the user painting their own firework image on the first computer.  Using a small, self-created program the user selects a color from a sidebar menu and begins drawing.  The program detects where the user has placed each pixel in a grid, and the data is then sent over the network to the main sketch computer.  The main computer translates the grid into individual particles ready to be launched off.  When the user steps up into the tent, the computer uses the Kinect to calibrate their center of volume.  Once this is determined, if the user extends their arm outside of a certain radius from their body the Kinect will detect that it is time to shoot the firework off.  In front of them the user will see their image being launched, as particles, into the night sky. At the same time, the user will hear the firework going up and launching all around them via surround sound that is connected to another computer over the network as well.  The intensity of the noise is directed in the direction that the firework is launched to simulate a real 3D experience.

The final result of our project is very exciting.  Inside the tent, the user receives an individualized, personal experience that they were allowed to create.  The 3D sound really pulls the entire event together, engaging the user and making it more enjoyable.

Everybody who played with our project at the show really seemed to enjoy
it, which is above all, what we had hoped for in making it. It was very
cool to see how each person tried to shoot their fireworks off, whether by
using two arms, flailing, or even kicking. Also, once a lot of people had
drawn their fireworks, since each one was added to our library, we got to
see what everyone had drawn by the end of the showcase. There were
fireworks ranging from random assortments of colors, geometric shapes,
fruits, and (as expected in an engineering building) video game characters
and dinosaurs.  It was definitely well received by all who used it.

Code: Kinect Presentation Speaker Server Custom Firework

Video:

Images:

Final Project: The Peripheral Arcade

Team: Justin McKeown, Henry Jonhson, Yaqi Ge

Concept:

Early on, we decided we wanted to strive for something simple and interactive, but really fun.  We wanted something that anyone could walk up to and get engaged immediately without the need for excessive instruction.  We were inspired by the previous year’s “Hands Free Super Three” game in terms of a selection menu, but we wanted to do something really unique with the games themselves.  We decided upon using three very popular and engaging arcade games:  “Pong”, “Space Invaders”, and “Defender”.  However, it wasn’t going to be very much fun to just stand there and play video games.  This is where our group’s creativity came into play and we decided upon making peripherals that would be used to play each game—Pong had a red and green paddle, Space Invaders had a complete ship that almost fully enclosed the player, and Defender had foam planes.  This new “twist” of playing classic arcade games was the core of our project, and it turned out to be surprisingly very successful.

Process:

We used GSVideo for our camera and Minim for sounds and music.

We took some pre-made programs from OpenProcessing for “Pong” and “Space Invaders” as a base for us to get started (Defender was entirely our own creation, but based heavily on the arcade game).  In order to use our peripherals to play the games, we needed some sort of motion tracking technology.  We found using color detection to work well:  We looked for specific RGB values (easily changeable as to adjust to light variations) that corresponded to the color we painted our peripherals.  If a certain threshold (normally a few thousand to compensate for noise) was found, we would determine the centroid of all of the pixels—which would become the centroid of the Pong paddle/ship on screen.  For Pong, we only used the Y coordinates.  For Space Invaders, we only used the X coordinates.  For Defender, we used both.

We altered Pong such that if enough red or green pixels weren’t detected, the AI would take over the paddle while printing a helpful message onto the screen.  If the player at any time wanted to jump in and play, he/she just had to move the paddle back into viewing range and he/she directly controlled that color paddle again.

We altered Space Invaders in the following ways:  The AI fires much more rapidly than normal to force the user to be more active moving side-side in the ship.  The ship the player controls fires automatically whenever NOT under a shelter, and also fires at an increased rate.  Due to this game being much longer than the other two, it often could get boring.  We added a background of the Earth as well as music from Space Invaders 91 (owned by the Taico Corporation) to keep the player entertained.

Defender was written from scratch, but borrowed a lot of functionality from Space Invaders.  We created a few different sprites for this project:  The player’s ship, type 1 and type 2 aliens, and human survivors.  The aliens and humans are both generated by a random number generator.  The type 1 aliens would search for their corresponding human (alien[8] searches for human[8]).  If their human exists, the aliens move towards their target, grab them, and take them off screen (creating a faster and deadlier type 2 alien that always chases the player).  If not, the type 1 alien chases the player.  This game is also the only one where the player can change their orientation by moving in the direction they want.  It is NOT tied to the orientation of the actual foam plane.

Results:

The results were very positive.  Almost every person told us “That was fun!” when they were done playing.  It was extremely rewarding to hear that, because it told us that we had achieved what we wanted to do all along:  Make something silly and fun so that the player has a good time.  Pong got a lot of use due to the player versus player aspect—even a little trash talking at one point (all in good nature, of course).  We were a little dissatisfied with how the color detection in Tishman Hall was a little unreliable, (especially for red) but the impact was only slight.  Now that we’ve presented the project once, we would definitely change the menu completely.  Some people had issues with waving their hands to choose their game (using the props themselves actually worked surprisingly well).  We overheard a suggestion about making a gigantic coin slot to select which game we wanted to play.  We all absolutely loved that idea, and it would keep with the arcade theme.

Code:

Project3

Video:  

http://vimeo.com/34038254

Images: