Project 2 :: Emotion Music

Posted by ahainen on November 2, 2010 Leave a comment (0) Go to comments

Andrew Hainen, Nikhil Mangla, Syed Wajahat Karim

Our project is a machine learning application that learns what proper emotion to associate with a song based on the characteristics of the song.

Conceptual Breakdown

Our idea was to have a computer react in a realistic fashion to music in a way that a human would react. With a human, if I’m walking down the street listening to my iPod and “Twist and Shout” by the Beatles starts playing, then I’ll hypothetically be happier, more upbeat, and cheerful. Oppositely, if “Smells Like Teen Spirit” by Nirvana comes on my iPod, then I might be more depressed, slow-walking, and down.

For a computer, we wanted to illicit these responses based on the certain types of songs it heard.

Technical Breakdown

To accomplish out concept, we are utilizing:

A machine learning experience based on jMIR’s jAudio library and the Weka library
Processing.org
GIF Animations

Primarily, we split a song of our choice into 10 second clips. Then, we run the clips through jAudio, which calculates 78 different numbers based on characteristics of the song. We then take those numbers, which have been compiled into a .arff file for the Weka Machine Learning Library, and use that to compare them. Once the the numbers have been compared, the program spits out “Sad” or “Happy”. From there, Processing.org will play a simple animation that is either “Sad” or “Happy”.

Final Output / Experience

Overall, we saw that the music data was very different in the “happy” and “sad” areas, which is what we were hoping for. There were approximately 78 paramaters the songs were being judged upon, most of which we were unsure of what they were measuring, but relied more so on the technical achievements of the jAudio Application.

Regardless of the data being quite different for each song, our data had mixed results. If a song we “thought of” as happy had a mild moment or didn’t meet all the requirements the program labeled as “happy”, then it would go to sad, even if it wasn’t a sad part. This “black or white” idea of categorizing music wasn’t the best implementation, but had to be relied upon for times’ sake.

Our original code included a “neutral” category of songs, but was skewing our results further than if we just had “happy” or “sad”. Furthermore, the concept of categorizing music into emotional categories is entirely opinion and debatable, and humans have learned how to do so over the course of their lives. We feel that using many more test samples for the program to learn our ideas of “happy”, “sad”, or “neutral” would greatly benefit the outcomes and results.

Documentation

Our final product can be seen in the areas below: