Robot Clarinet

This clarinet-playing robot was built by a NICTA − UNSW team for the Artemis orchestra competition in 2008, where it was the winner. The contest rules require embedded device robots, with mass less than 20 kg, that play unmodified musical instruments.

Why a clarinet robot? No, the aim is not to replace human musicians. The robot serves three functions:

• It is an interesting challenge to engineers and scientists to understand and to implement some of the complicated things that humans do when playing music. This is the reason behind the competition.
• It is interesting for the music acoustics lab to see how well we understand clarinet playing and it provides us with a very patient and reproducible 'volunteer' for experiments.
• Currently, the robot is doing experiments to correlate the transients, reed motion and sound produced with fingering, mouth pressure, lip force and damping and mouth geometry. It can cover a large region of parameter space overnight and tell us the results in the morning. (Preliminary report is here.)

Duet with Deborah de Graaff

At a seminar in Music Science, the robot played a duet with eminent clarinet soloist Deborah de Graaff.

The clip below shows an archival 'performance' when the robot was put together for the very first time. We had just received some of the last components and assembled it to try it out. For simplicity, we ran it that night with the relays in the open air, and you can hear them clicking on every fingering change. (In the duet video above, they are encased in a box to reduce noise.)

Although the robot had just been 'born', it already knew how to read music files, and its coordination was pretty good. So it skipped some of the stages that a human musician would go through. On the other hand, the Mark 1 version is deaf -- it doesn't yet have feedback.

John Judge	project manager, broad range of tasks	Senior Research Engineer, NICTA
Mark Sheahan	embedded software, student leader	Student, Biomedical Engineering/Computer Science, UNSW
André Almeida	signal processing	Postdoctoral fellow, Physics, UNSW
Peter Chubb	music software	Senior Research Engineer, NICTA
Kim Son Dang	mechanical design	Mechanical Engineering, UNSW
Jean Geoffroy	mechanical-acoustical	Student, Physics, UNSW
Jay Katupitiya	mechanical design	Mechanical Engineering, UNSW
Paul Santus	mechanical and control	Student, Physics, UNSW
Andrei Skougarevsky	mechanical	Physics, UNSW
John Smith	electronics	Physics, UNSW
Joe Wolfe	music acoustics	Physics, UNSW
Pritipal Baweja, Ian Cassapi, Andrew Higley, Radha Kottieth Pullambil	mechanical workshops	UNSW

The clarinet robot began when John Judge from NICTA visited the Music Acoustics Lab and the Mechanical Engineering School in November 2007 and suggested a collaboration to enter the competition. Both were keen to collaborate. For Mechanical Engineering, the project represented an interesting design challenge on a short time-scale. (The photo shows Jean, Joe, Mark and John.) The pump and controller were kindly supplied by ResMed.

As you can tell, this is our first attempt at a music robot, and it was built in six months. So many of the features that we planned to put in are still absent. For both NICTA and the Music Acoustics lab, the robot is now a platform for research on a number of issues relating to musical performance and the player-instrument interaction. And no, we're not aiming to put humans out of a job. For us, the robot is a complementary part of our research into clarinets and how to play music badly or well, and what makes the difference. See An introduction to clarinet acoustics.