Our technological prowess is increasing every day, but the machines we use are still limited by the knowledge that the designers put into them. Researchers have started experimenting with a branch of computer science called “machine learning,” where computers can learn and adapt. The name evokes sinister Terminator imagery, but researchers at UC San Diego have been using machine learning to develop the most advanced music search engine ever created.
The researchers are using the same methodology in their system as hospitals use to train new doctors.
“When doctors learn how to diagnose diseases, they are trained to find patterns in patient data that indicate which disease the patient has. With many patients, patterns begin to emerge,” said Gert Lanckriet, a professor of electrical engineering at the UC San Diego Jacobs School of Engineering.
People identify music by its characteristics, just like doctors identify diseases based on symptoms; is it jazz, rock or dubstep? Is it happy or sad, slow or fast, energizing or relaxing, piano or saxophone? The researchers started with a library of about 13,000 songs in their program, called Herd It, and started labeling each song with tags like those above. The program would then analyze patterns in the music and associate those patterns with the tags.
As more songs were tagged, Herd It began to learn for itself what patterns constituted each genre or style, and it began to make its own tag suggestions. The researchers would simply confirm or reject the program’s suggestions. The more data Herd It received from confirmations and rejections, the better it became at suggesting tags.
“The algorithms continuously adapt,” said Doug Turnbull, a graduate student working in Lanckriet’s lab on the project. “People labeled music for us in the fun context of a game. People label the song with any tag [they think is appropriate] and we use the most common ones.”
The music streaming service Pandora already does something similar. However, Pandora pays music experts to label songs. Each expert at Pandora takes about 30 minutes to tag each song by hand, making Pandora’s process very expensive. As such, Pandora has only about 900,000 songs in their library, compared to the near 20 million in the iTunes library.
Herd It is an AI (artificial intelligence) that can tag any song it encounters in a matter of seconds, using plain english terminology to describe genre, feelings and settings. It does the same thing that paid musicologists do, but cheaper, faster and in simpler terms. The program works by analyzing the waveform of a song and identifying patterns in the frequencies that are indicative of each tag.
The machine learning process is a continuous feedback loop. Users confirm or reject the program’s predictions, the machine learns something, and then it refines its predictions for the next guess. Every little bit of input from a user refines the algorithms’ accuracy on its future guesses.
Once the program has correctly identified songs, it adds the song name to a database with its related tags. Currently, Herd It uses between 200 and 300 tags to identify music, but it will soon use close to 1000.
According to the researchers, the machine learning was not the most impressive part of this project. Instead, the most unique feature was the use of crowdsourcing to create training data for a computer program, where humans are teaching a machine.
The final goal is for Herd It to not just identify a music type, but to identify the preferences of a listener, even if the listener is not consciously aware of those preferences. By analyzing listening patterns, Herd It will be able to give you the best possible match for your next song.
For Luke Barrington, founder of the UC San Diego Computer Audition Lab where Herd It was created, the advantage of Herd It is that people can search for music in natural language, rather than technical jargon.
“[Herd It] describes any piece of music in human words,” Barrington said. “You give [Herd It] words, it gives you music.”
HUDSON LOFCHIE can be reached at science@theaggie.org.
[…] Crowdsource for truly unique musical experiencesThe AggieOur technological prowess is increasing every day, but the machines we use are still limited by the knowledge that the designers put into them. Researchers have started experimenting with a branch of computer science called “machine learning,” where … […]