Q&A: Tod Machover on personal music

Journal name:
Nature
Volume:
466,
Page:
320
Date published:
DOI:
doi:10.1038/466320a
Published online

The inventor and composer, whose group at the Massachusetts Institute of Technology's Media Lab developed the technology behind Guitar Hero, has electronically customized instruments for musicians from Prince to Yo-Yo Ma. As Machover prepares for the world premiere of his robotic opera Death and the Powers in Monaco in September, he explains how his interactive performance techniques might lead to personalized therapies.

Death and the Powers

Composed by Tod Machover. Libretto by Robert Pinsky. Directed by Diane Paulus
25–26 September 2010, Opéra de Monte-Carlo, Monaco. See http://go.nature.com/9bQ6B7

What is the opera about?

Mortality. I know my parents well, but how will their grandchildren know them? What can a dying person pass on to future generations? I researched people such as US entrepreneur and inventor Ray Kurzweil, who is trying to live forever; people who are trying to preserve themselves on ice; and people with locked-in syndrome, who have almost complete paralysis and cannot communicate but are not dead.

The opera tells the story of Simon Powers, a man who wants to remain in the world after he dies. So he invents a system to download everything about himself — his legacy, personality and memories — into his environment. At the end of the first scene Powers disappears, and the stage set becomes the main character.

How do you use robotics to portray Powers after his death?

G. SPIRO

The system into which Powers has transferred his intelligence has four elements: two kinds of robots, animated bookcases and a musical chandelier. The bookcases have LED displays that look like the spines of thousands of books; as they move and change texture, they allow Powers to express himself. A swarm of nine 'opera bots' acts as a chorus. They are wireless, intelligent and graceful; white and smooth with lights all over. The other category of robots is cruder. They look like strange animals and react nervously to the characters onstage.

You've described the chandelier as the first use of sonic animatronics. What is it?

It looks like an upside-down Venus fly-trap plant, with wings of rounded metal that are criss-crossed by dozens of Teflon strings that extend about 12 metres when it opens. It is a robotic musical instrument in which the strings are vibrated using electro-acoustic signals. We can control the harmonics and produce rich sounds, like low piano notes, as well as pulsations and vibrations.

How will the actors interact with the stage?

We draw on a new technique called disembodied performance to animate the onstage environment using the offstage voice, breathing and arm movements of James Maddalena, who plays Powers. One big question is how much he will be able to control while he is singing and reacting to other performers. We also want to measure his muscle contractions, pulse, posture and other things he can't consciously control.

Why did you choose the medium of opera?

With the world being fragmented and busy, there is something powerful about going to a public place and experiencing something that is never going to happen again. Opera also attracts me because it is a hybrid, pragmatic form that has always sought to pull every possible discipline into its orbit — much like the MIT Media Lab encompasses so many areas of human exploration. Opera may seem to be a traditional form, but it is ripe for, and open to, reinvention.

C. R. DE MASSY/V. BINA

How has opera inspired your research?

The first opera I did was VALIS, based on Philip K. Dick's novel. It was staged in the lobby of the Pompidou Centre in Paris in 1987 with almost no scenery. I wanted to rehearse the pit orchestra like a rock group, for hours every day, but I couldn't afford that much time. So I reduced the orchestra to two musicians. And that is when I invented hyper-instruments — acoustic instruments that are measured and sensed electronically in ways that affect the sound, to maximize the variety of sound coming from each one. These instruments, based on commercially available technology, eventually led to the video games Guitar Hero and Rock Band.

How do you think music affects us?

I am optimistic that music can play a part in combating disease — both physical and mental. Music therapy is an old profession, but we are just beginning to understand the science that will allow treatments to be shaped to an individual. I am adapting my hyper-instruments so that they can be adjusted to the skills or limitations of each person.

I also believe that in 50 years we will be able to customize each piece of music, and the way that it is played, to produce a precise emotional and physical reaction — hopefully a beneficial one — that will be unique for each of us, in each situation, in each moment. This 'personal music' has never existed before, but I intend to help lay the groundwork for it.

Additional data