Hands Free Super Three

by Joe Mitchell, Landon Smith, and Stephanie Boxold.

Hands Free Super Three utilizes computer vision techniques to present three classically themed games played hands free (sans controller)!  The main menu displays options to select a game by waving your hands or other appendages over one of the three buttons.  Using the open source computer vision library (OpenCV), face detection, motion detection, and various other techniques are implemented to create an immersive, but hands-free, gaming experience.

The first of the Hands Free Super Three is Monster Attack, a game reminiscent of scenes from old-time monster movies such as Godzilla or King Kong, but with you as the monster! Fighter planes attack from all sides, and you must swat them down before they drop too many bombs on you. It’ll be second nature to you though, as your face is replaced by a monster’s visage!

In Block Busta, you control a paddle as you bounce balls against an array of bricks. It’s a well-known game, but this time, you’re controlling the paddle with your face – literally! The game detects the position of your face, and uses your very face to bounce the balls. It’s a good thing there’s no force feedback, don’t you think?

Face Invaders, as you might expect, is an adaptation of the truly classic game Space Invaders, but again controlled by your face. Dodge lasers and fire your own in retaliation from your ‘faceship’. Your lasers are triggered by your movement, so be sure to keep moving (waving your arms, legs, anything!) and fly your faceship to victory.

Hands Free Super Three was developed in ‘Processing’, utilizing OpenCV (an open source computer vision library) for face detection. We created our own method for the motion detection used in the main menu, Monster Attack, and Face Invaders. Sound effects are played via the Minim audio library. Everything used in the creation of this project is open source and available to anyone to use/play with. Our project utilized a simple webcam, like the ones built in to most computers these days.

There were many technical challenges to overcome developing our games. The computational overhead of doing things like motion detection and face detection is significant. To make the games playable, we created multiple threads to run processes asynchronously letting the detection processes run continuously in the background while the game animation plays uninterrupted.

The installation at the exhibit presented additional challenges with camera placement and lighting. The choice to put the camera at the bottom of the screen, in addition to being a departure from the development environment, made face recognition by the software more difficult (it was often necessary to aim your face at the camera to get it to pick up your position). Additionally, the lighting varied at times which again played havoc with face detection, but also with our method of movement detection. This was particularly notable in Face Invaders, where it became harder to fire your laser as the ambient light decreased. These issues could all be resolved by tweaking various thresholds in the code, but doing so on a case by case basis would be cumbersome. A solution might be to sample the images being captured by the camera to get a general idea of the overall lighting conditions and adjusting the light sensitive variables by some factor based on that sample.

Ultimately, our games were fun, and brought many smiles to many faces. We are glad that so many people got to enjoy our work! The video game sphere is evolving in some interesting ways right now, with motion gaming being at the forefront of the evolution. Nintendo really changed the environment with their Wii and accelerometer based input, but now Microsoft has demonstrated you don’t need a controller with Kinect. Our project embodies this new approach to controller free gaming, but with a more approachable, indie-style flavor. There are a lot more people with computers with webcams than people that have XBox 360s with Kinect controllers, and we think there is a lot of room for developers to make games in this vein, much like social gaming sites (like facebook) have brought gaming to the masses that don’t have any gaming console.

Feel free to download the code we created from the link below. If you enjoy it, please leave a comment! Thanks!

Developers:
Joe Mitchell
Landon Smith

Artist:
Stephanie Boxold

Code: Hands Free Super Three

Video Links:

Landon demonstrates all three games:
http://www.youtube.com/watch?v=thLus5pmqOY

Here are some pics of the “final” project (from a laptop, not the installation):

Main menu as seen from my living room.

Santa is a monster?

Monster Attack with Santa Claus head!

Alien!

Monster Attack: Alien Head Edition

Face Paddle!

Block Busta, where your head is the paddle.

Flying my FaceShip

Beware of the aliens in Face Invaders!

From the installation exhibit:

Menu

The Hands Free Super Three Menu

Alien Monster!

Landon plays as the alien in Monster Attack

Here are pics of the project in development:

Main Menu for 'Hands Free Super Three'

Main Menu for 'Hands Free Super Three'

Monster Attack!

The Monster and the Plane from Monster Attack

Block Busta in Development

Block Busta

Face Invaders

Face Invaders

Leave a comment

2 Comments.

  1. Your titles are hilarious. How are you dealing with the issue of firing from the “faceship”?

    • The plan is to use movement detection in certain regions to trigger the ship’s laser. I am still working on implementing new movement detection code so I can reuse the same bits for the various cases where we have to do movement detection (already did this with face detection). We’ve implemented multi-threading to massively improve performance!