Joshua Wehrly, Steve Wishnousky, Cassandra Yaple
Our project transforms the physical interactions of the users into auditory sensory feedback. The user enters the environment either aware that there may be something there, or just by haphazardly entering the space and triggering sound. In either way, the experience from then on is exploratory, as the user attempts to figure out how they made the sound, and how they can most interestingly trigger the sound. They can explore the soundscape they create either alone or with others in a collaborative experience.
Our project uses the Microsoft Kinect device as a sensor because of it’s unique attributes as being both a regular camera and a depth camera, the latter being the most important one for this project. We used the depth camera to set a specific range that we determined would be the wall, and then broke down the whole region into squares. We chose to have 21 squares in total, since we wanted the grid to be 7 across and 3 down. We chose their orientation since there are 7 notes in the typical scale, and in our experimentation it felt most natural to have one zone for the lower body by the feet, one for the middle that you could easily reach by simply outreaching your hand, and one that you had to reach for. When the range finder detects anything in the range, it triggers the note that the square is assigned to. In order to make this work correctly, given the project should be able to detect multiple bodies and play multiple notes, we threaded our program so that one thread would be responsible for detection, and new threads responsible for playing the sound would be released upon request.
We had a couple issues with actually getting the Kinect device to work, given that it was newly released and the open source drivers were not at all stable. We ended up managing to work the device by using Processing on Mac OSX software.
Our project was successful in being the interactive and fun experience we hoped for, as many people were drawn in and found the experience both interesting and enjoyable. Some users took a while to figure out how to control what sounds were made, while others picked it up quickly and may even done things that we had not thought of. In particular, Professor Robert Adams, an Assistant Professor in Architecture at the University of Michigan, thought it was very interesting and would like to commission the piece for an exhibit at UC Berkeley this Janurary.