8.2 A Framework for Future Research

This section suggests some of the possibilities for extending the Conductor’s Jacket project and making use of its implications in future research.

8.2.1 Extensions to the Conductor’s Jacket Project

One way that the current findings of this project could be applied would be to use similar methods and sensors as were used here to research the link between music and the emotions. While many theorists and musicians have acknowledged that music is a very powerful medium for emotional expression, very few have looked at physiological correlates to the musical/emotional experience. Charles Darwin may have been the first to describe the perception of music in behavioral terms:

"Music has a wonderful power, as I have elsewhere attempted to show, of recalling in a vague and indefinite manner, those strong emotions which were felt during long-past ages, when, as is probable, our early progenitors courted each other by the aid of vocal tones. And as several of our strongest emotions – grief, great joy, love, and sympathy – lead to the free secretion of tears, it is not surprising that music should be apt to cause our eyes to become suffused with tears, especially when we are already softened by any of the tenderer feelings. Music often produces another peculiar effect. We know that every strong sensation, emotion, or excitement – extreme pain, rage, terror, joy, or the passion of love – all have a special tendency to cause the muscles to tremble; and the thrill or slight shiver which runs down the backbone and limbs of many persons when they are powerfully affected by music, seems to bear the same relation to the above trembling of the body, as a slight suffusion of tears from the power of music does to weeping from any strong and real emotion." It would be a straightforward job to use the equipment and sensors of the Conductor’s Jacket for audience members at a concert; very few adjustments or arrangements would need to be made at all. The lack of motion of a seated concert-goer would also mean that there would be more promise for the integrity of the signals such as heart rate, skin conductance, breathing, etc. since there would be no significant motion artifact. A jacket-like device might be very useful for numerous research by psychologists on music perception and experience. It could be used to find out if the audience agrees that it has contagiously felt what the performer attempted to convey. It would also be interesting to look at the respiration signals of conductors under other circumstances such as sitting, thinking about a particular piece of music, and listening to a particular piece of music. In such cases, when motion artifact is not at all an issue, it would be interesting to look at the heart and breathing rates.

Another extension of this work that has been discussed from the beginning of the project would be to study the effects of conducting on the musicians of the orchestra. One question that the Conductor’s Jacket did not address is the relationship between the indications of the conductor and the final creation of the sound. It would not be technically difficult to redesign the jacket architecture to be practical to be worn by several instrumentalists; studying how their behavior changes based on the gestures of the conductor would be fascinating for all sorts of reasons. Perhaps the behavior of an orchestra relative to a conductor is the kind of model that should be considered for future gestural instruments; if we can model this relationship reasonably well, then gestural instruments might seem more ‘responsive’ and ‘human.’

Another extension of the current work would be to evaluate the Gesture Construction system by having one of the original conductor subjects perform with it. An interesting experiment would be to hold a session in which one of the original subjects wears the Conductor’s Jacket and conducts an ensemble simultaneously. With a piece such as the Bach Toccata and Fugue in D minor, the computer system could play its performance directly to a recording, while the live musicians perform and record the same piece. If the computer version of the piece resembled the human version, then this system can be said to have been successful.

Additionally, it would be interesting to interview conductors before and after a data-gathering session to find out their opinions about the piece and its interpretation. It would be particularly valuable to have a conductor review the video and signal recordings of the session along with the researcher, so as to register his or her opinions about the data and its significance. I had intended to do this during the course of this project, but it was rarely possible due to the schedules of my subjects. Perhaps with a greater range of subjects it might be possible to arrange for some time in which to do this kind of work and give the conductors an opportunity for self-report.

Another way to take this work would be to talk with composers who work primarily in electronic media to find out their opinions regarding the existing real-time controllers for electronic music. It would be important to find out what aspects of expression they wish they could control, and which expressive features they wish would work better. Then it might be possible to design systems that work better for the very people who are dedicated to electronic music as an art form. It might be worthwhile to request one of them to revisit the analyses developed in the Conductor’s Jacket project and see if they think the features are useful or interesting.

Finally, I think it would be fun to use the results of the Conductor’s Jacket system to explore the complex relationships between the concepts of instruments, ensembles (orchestras), players, and pieces. Robert Rowe explained that electronic systems could either follow the instrument- or player- paradigms, where they are treated as an instrument to be controlled or a partner in the creation process. Pete Rice extended this distinction to include the paradigm of the composition. The Conductor’s Jacket system could be interpreted as an instrument, orchestra, or piece. For example, it might be fun to write a piece where the performer competes with an invisible agent for ownership of control over the sounds. Or the Jacket itself might even have its own identity as a player that pits it against its own performer. The free exploration of the possibilities between the piece, player, instrument and ensemble distinctions could generate a new category of music performance.

8.2.2 Implications from the Conductor’s Jacket for Other Work

From the beginning of the Conductor’s Jacket project it was hoped that there would be important implications for it in areas far beyond its initial musical test-bed. For example, the real-time synthesis developed for this project made it clear that there is a big need for non-linear, dynamic filters that have knowledge of features over time. Some of the filters that were written for processing the conducting data could be useful in many other application areas.

Also, it’s been suggested by many people that the Conductor’s Jacket is potentially very promising as tool for teaching and practicing conducting skills. For example, I’ve been approached by a professor of conducting at the University of Arizona, who has asked me to develop a version of the jacket for his class. Of course building a conductor’s jacket to teach students how to gesture like a conductor is only one part of the issue; then there is the notorious difficult problem of how one obtains conducting experience. How does one "practice" conducting without an orchestra? Future versions of the jacket could be very useful for this in the sense that they might simulate the orchestral sound much more realistically than the current version of the jacket does. This could be a great help to certain conductors, who otherwise have to cajole or pay an ensemble of musicians when they want to get experience.

Another idea outside of music would be to redesign the jacket so as to be an enabling digital interface for either handicapped or remote users. For example, Marvin Minsky mentioned to me on more than one occasion that he would like to have an armband of EMG sensors around both forearms so that he could touch-type on any flat surface without an actual keyboard and have the wearable system record the characters as he types them. It’s also well-known that many mechanical prosthetic devices for people who have lost limbs are controlled by means of EMG signals; it might be possible to set up a device that uses different muscle groups (such as those in the upper shoulder area or face) to type characters. It was also brought to my attention that in certain sports such as golf, where the activation and force of different muscle groups is critical and well-defined, a device like the Conductor’s Jacket could be used to help people refine and improve upon their swing, stance, and follow-through. Even medical conditions such as motion disorders might be ideally suited to be analyzed with a device such as the Conductor’s Jacket. There is also a precedent in the field of character animation for the modeling and synthesis of expressive movement; it might be that a jacket-like device could be used to analyze the movements of real creatures in order to understand and generate ‘realistic-looking’ versions of those behaviors for the screen.

Finally, there is a possibility for a whole class of wearable musical instruments for the general public, including musical clothes to be worn by children to teach them about music through kinaesthetic activities. Also, it is conceivable that a wearable device could be built to allow anyone to conduct a stereo recording from the comfort of their home or even their couch. Lastly, the work that has been presented here might best be used to interface with other, existing systems for music, such as Manfred Clynes’ SuperConductor software package.

  Chapter 8.3