Formulated to respond to the issues raised by the Digital Baton, the Conductor’s Jacket project was begun in the spring of 1997. The basic premise of the project was to build a device to sense as many potentially significant signals from a working conductor as possible without changing his behavior. We chose to focus primarily on physiological indicators, since Healey and Picard had shown that they correlated strongly with affective states. After designing and building the system, we ran a series of data collection sessions with student and professional conductors in the Boston area.
This chapter describes the physical components in the Conductor’s Jacket system, including the wearable jacket, the sensors, and the associated sampling hardware. The Conductor’s Jacket system was not a monolithic, single entity, but rather a collection of different designs and architectures that were chosen and adapted for a variety of conditions. By the end of the project I had developed four different jacket styles, eight jackets, various sensor configurations, and a reconfigurable data acquisition environment on two computers. Details about the background investigations, implementation, and data collection experiments are given in this chapter.
The concept for the Conductor’s Jacket was first suggested by Professor Rosalind Picard in November 1996; at that time she and I brainstormed an image of a conductor in a tuxedo jacket, appearing completely normal to the outside world. However, we imagined that the jacket would be completely wired up with a range of physiological and positional sensors, even accompanied by GSR sensors in the shoes and possibly devices in the podium and music stand. At the time we also envisioned a completely wireless design with a wearable computer embedded in the jacket to take care of all computational functions. In our idea, a conductor would be free to conduct rehearsals and concerts in this jacket without any disturbances or distractions for the audience or orchestra, while meanwhile providing data on his gestures and affective states.
Surprisingly enough, many of those ideas were practical and possible to implement, and we did, in fact, build much of the system that I described above. Some ideas turned out to be problematic, however, such as the wearable computer. This was particularly due to the fact that our conductor subjects were generally not comfortable with using computers, and also that they had extremely limited time and attention to spend on the computer and sensors when they had an entire orchestra rehearsal to run.
3.1.1 Preliminary Investigations
The Conductor’s Jacket project began with investigations into sensors and data acquisition methods. I started by evaluating the usefulness of EMG sensors for conducting gestures. During this time Lars Oddsson of the Boston University NeuroMuscular Research Center graciously explained the issues with EMG sensors and signals, and allowed me to use the sensors and acquisition hardware in his lab to run some pilot studies on myself. I collected data on three different muscle groups during a variety of conducting gestures, and found a number of promising results. First of all, it was obvious from the first attempt that I would be able to recover beat information from these signals; all of the major muscle groups of the upper arm registered clear peaks for clear beat gestures. Also, the amplitude envelope of each peak seemed to reflect the force profile of the muscle in the execution of the gesture. However, there were noticeable differences between different muscles; the biceps tended to give clear spikes at the moment of the beat, whereas the triceps and lateral deltoid (shoulder) muscles provided a smoother rise and decay with secondary modes on either side of the beat. This is demonstrated in the figure below, with the biceps signals in green, the triceps signal in red, and the lateral deltoid signal in blue:
In this example, six downbeat gestures are shown in succession,
with three EMG signals from the upper arm. The results of my limited pilot
study indicated that electromyography sensors might yield promising results
with conductors. Other similar trials were done with sensors for respiration,
heart rate, galvanic skin response, temperature, and position.