Robot Clarinet

This clarinet-playing robot was built by a NICTA − UNSW team for the Artemis orchestra competition. The contest rules require embedded device robots, with mass less than 20 kg, that play unmodified musical instruments.

News from Athens: We won the 2008 competition!

Why a clarinet robot? No, the aim is not to replace human musicians. The robot serves two functions:

• It is an interesting challenge to engineers and scientists to understand and to implement some of the complicated things that humans do when playing music. This is the reason behind the competition.
• It is interesting for the music acoustics lab to see how well we understand clarinet playing and it provides us with a very patient and reproducible 'volunteer' for experiments.

Duet with Deborah de Graaff

At a seminar in Music Science, the robot played a duet by Tartini with eminent clarinet soloist Deborah de Graaff. This is a low resolution movie. Here is a high resolution video of the performance (warning: large file, slow download).

The clip below shows a 'performance' when the robot was put together for the very first time. We had just received some of the last components and assembled it to try it out. For simplicity, we ran it that night with the relays in the open air, and you can hear them clicking on every fingering change. (In the duet video above, they are encased in a box to reduce noise.)

Although the robot had just been 'born', it already knew how to read music files, and its coordination was pretty good. So it skipped some of the stages that a human musician would go through. On the other hand, the Mark 1 version is deaf -- it doesn't yet have feedback. In the duet video above, Deborah is keeping time with the robot, without any of the usual gestures (head nods, breathing, etc) that musicians use for synchronisation. Deborah said that this was the hardest thing to do.

John Judge	project manager, broad range of tasks	Senior Research Engineer, NICTA
Mark Sheahan	embedded software, student leader	Student, Biomedical Engineering/Computer Science, UNSW
André Almeida	signal processing	Postdoctoral fellow, Physics, UNSW
Peter Chubb	music software	Senior Research Engineer, NICTA
Kim Son Dang	mechanical design	Mechanical Engineering, UNSW
Jean Geoffroy	mechanical-acoustical	Student, Physics, UNSW
Jay Katupitiya	mechanical design	Mechanical Engineering, UNSW
Paul Santus	mechanical and control	Student, Physics, UNSW
Andrei Skougarevsky	mechanical	Physics, UNSW
John Smith	electronics	Physics, UNSW
Joe Wolfe	music acoustics	Physics, UNSW
Pritipal Baweja, Ian Cassapi, Andrew Higley, Radha Kottieth Pullambil	mechanical workshops	UNSW

The clarinet robot began when John Judge from NICTA visited the Music Acoustics Lab and the Mechanical Engineering School in November 2007 and suggested a collaboration to enter the competition. Both were keen to collaborate. For Mechanical Engineering, the project represented an interesting design challenge on a short time-scale. (The photo shows Jean, Joe, Mark and John.) The pump and controller were kindly supplied by ResMed.

As you can tell, this is our first attempt at a music robot, and we only began in November 2007 for a contest in June 2008. So many of the features that we planned to put in are still absent. For both NICTA and the Music Acoustics lab, the robot is now a platform for research on a number of issues relating to musical performance and the player-instrument interaction. And no, we're not aiming to put humans out of a job. For us, the robot is a complementary part of our research into clarinets and how to play music badly or well, and what makes the difference. See An introduction to clarinet acoustics.