Published online 28 May 2004 | Nature | doi:10.1038/news040524-10


Smart music system skips to chorus

Song analysis could replace fast forward button.

The programme highlights the chorus and other repetitive areas of music.The programme highlights the chorus and other repetitive areas of music.© Masataka Goto

Sick of hitting fast forward to find your favourite bars of a song? A Japanese researcher has invented a new programme that can jump straight to the chorus or verse.

The prototype programme, dubbed the SmartMusicKiosk, was exhibited at the meeting of the Acoustical Society of America in New York City on Tuesday. It was invented by Masataka Goto of the National Institute of Advanced Industrial Science and Technology, Tsukuba.

Blasting pop classics such as My Heart Will Go On by Céline Dion, and Jon Bon Jovi's You Give Love a Bad Name, Goto showed how the SmartMusicKiosk skipped the filler and zipped straight from chorus to chorus. "I really like this music," he said.

Goto hopes the SmartMusicKiosk will ultimately enhance conventional CD and digital music systems, which only recognise the gaps between songs. Frustrated listeners are left pressing fast forward and rewind buttons to reach the choice refrain.

To automate this, Goto had to design a system that could recognise the structure of music, and then make out repeated choruses. In an advance on other such methods, Goto's takes into account the fact that the key, lyrics, and accompaniment often change from one chorus to the next.

The SmartMusicKiosk breaks down the song into twelve different pitches ranging from C, C#, D and so on up to B. It then sums together the power of the frequencies at each pitch, so a high B and a low B are added together. It analyses how these twelve pitches change over time through the song.

The algorithm then looks for matching patterns in the pitch structure and selects which is most likely to represent the chorus, assuming this is the longest and most frequently repeated section, and often broken into two similar refrains. The analysis takes about one minute for a four-minute tune, and gets the chorus right about 80% of the time.

For listeners, the programme produces a Music Map on screen, which reveals repeated segments of the song, and has a Jump to Chorus button alongside the regular play and pause keys. "You can listen to any part of a song whenever you like," Goto says.

Goto hopes that listening stations in record stores will be first to adopt SmartMusicKiosk, because shoppers are impatient to hear the chorus of a song to judge whether they want it. He says he has already been contacted by companies interested in commercialization.

Although it is designed for pop songs with clear choruses, the SmartMusicKiosk can also pick up recurring sections in classical music, Goto showed. However, experimental music may prove more of a challenge.