Cellist Katinka Kleijn performed both halves of a duet Sunday night. Her hands played the cello, and her brain, hooked up to a headset that detects cerebral electrical signals, played itself. Kleijn has been playing the cello for 35 years. Her brain was a little less experienced.
“Intelligence in the Human Machine,” the cello/brain duet, explored the relationship a performer has to the music she’s playing. During the performance, at Chicago’s Cultural Center, Kleijn wore an Emotiv EPOC, a neuroheadset with 14 sensors that attach to the scalp and detect brainwaves. In front of her, a laptop flashed a word and a few measures of music. She then played the music on her cello, interpreting the word onscreen. At the same time, her brainwaves, translated to audio, changed sounds as she reacted to the word.
(MORE: The Serious Science of Creating Music from Brain Waves)
“She might have a very peaceful melody, but she’s received the word violent, so through this peaceful melody, she has to emote violence,” Daniel Dehaan, who composed the music and collaborated with Kleijn and sound engineer Ryan Ingebritsen on the performance, told TIME. “She might do some terrible things to that melody.”
Some words were harder to interpret than others. “Remorse” proved particularly challenging.
“I guess I don’t have a lot of remorse,” Kleijn told TIME.
(PHOTOS: Odd Couples: 10 Weird Christmas Duets)
Foot pedals controlled when the words and music changed. Kleijn could tap the left pedal, and the word would change from “failure” to “neutrality,” and the same bars of music would sound completely different, reinterpreted for the new word. Similarly, tapping the right pedal swapped out the music for a different tune.
When a new word came on the screen, Kleijn “would just try to experience it, focus on it, think about it, and then interpret it in the musical element,” she explains.
It was a balancing act for her whole body.
“Not only is Katinka playing the cello, but she is also, in a sense, playing her brain waves, emphasizing what’s going on in her brain while she’s performing,” Dehaan says.
Those brainwaves don’t usually make noise, at least not any noise that humans can hear. But the brain fires off electrical impulses that, when recorded on an EEG machine, look an awful lot like sound waves. Using special software, Dehaan and Ingebritsen translated Kleijn’s brainwaves to audio waves, which onstage became a symphony of pops, clops and pitter-patters. They sped up when Kleijn got excited, sounding like horses galloping; when she was more meditative, they slowed to the pace of water dripping from a faucet.
Though the collaborators had an idea of what the performance would sound like, they didn’t know exactly how it would come together. But just by pinpointing the link between action and response, no matter how it sounded, made the piece a success.