From March through July 1999 I implemented several pieces for the Gesture Construction, including both etudes (studies) and compositions to be performed. Many of the etudes were written to test or experiment with some component of the system and were not intended to be practiced or performed. Five of the completed works are described below.
This etude was written to provide auditory feedback to the performer to indicate the state and health of the system right before a performance; this is similar to the function of ‘tuning,’ which classical musicians do on-stage right before beginning a concert. In the classical tradition, tuning is done not only to make sure that all the instruments match the same reference pitch, but also for the musicians to play a little bit to test the state of their instruments, with enough time to make a few minor adjustments. This etude plays out repeated notes sonifying the raw output of each successive sensor from 0-7, mapping pitch and volume to the signal output. The sounds clue the performer in to the state of the system by sonifying the range and responsiveness of the sensor; if the sensor is not behaving as expected, the performer will notice an audible difference from her expectation.
6.5.2 Etude 2: One-to-One Relationship
The second etude has one voice that is mapped to the filtered output of the right biceps EMG sensor; its volume, timbre, and pitch are modified simultaneously (although with slightly different algorithms) by the signal. If the muscle is not used, no sound plays; as the muscle contracts, a sound grows in volume and increases in pitch and brightness. This very direct relationship provides both the performer and the observer with an immediate sense of the profile of the muscle tension signal.
6.5.3 Etude 3: Beats, Cutoffs, and Crescendo/Diminuendo on Sustain
In the third etude the right biceps EMG generates beats and cutoffs for a set of chords. Once a chord is initiated it is sustained indefinitely (up to a maximum time of 18 seconds) until a beat again cuts it off. While a note is playing, the left biceps value scales the sustained volume of the note (channel volume); tensing the muscle make the note get louder, relaxing it makes the note get softer.
6.5.4 Bach’s Toccata and Fugue in D minor
The first full performance piece that was implemented with the Gesture Construction was intended to be a realistic simulation of conducting. The mapping strategy applied several of the features that were learned from the conductor analysis, including the use of the left hand for expressive variation, the one-to-one correlation between muscle tension and dynamic intensity, the division of labor between biceps, triceps, and forearm, and the link between respiration and phrasing. While the system does not completely synthesize all of the features, it was intended to stand as a proof of concept that the physiological signals of a musician can be used to drive the expressive aspects of a real-time performance. The piece that was chosen was the Toccata and Fugue in D minor, a composition for the organ by J.S. Bach. I created an orchestration for the piece similar to the one that Stokowski wrote for the Philadelphia Orchestra and performed in Disney’s Fantasia.
In the performance of this piece the action of the right biceps muscle determines the beats, tempo, beat volumes, and cutoffs. The right forearm gives articulations; sustained contraction of the muscle yields longer notes and therefore a legato quality, whereas shorter contractions of the muscle yield shorter, notes with a staccato quality. The use of the left biceps muscle causes octave doublings to fill out the bass and the aggregate values of the left arm muscles versus the right arm muscles determines the panning of all the voices. The piece was implemented as a full score and once initiated, cannot be reversed. It can be paused in the middle if the performer does not generate new beats, although this direct-drive model is susceptible to noise.
The final piece written for the Gesture Construction is the "Song for the End," a composition of mine that uses some techniques from the North Indian Khyal vocal style, where the set of notes is predetermined but can be performed improvisationally with any rhythm. (This is closely related to the Western classical concept of ‘recitative.’) The opening section is a melismatic vocal passage in which the notes are sequenced in a particular order but every other facet of their performance – start time, volume, accent, length, and timbre are under the direct control of the performer. This is followed by a fixed sequence in which different muscles can raise and lower the note velocities, channel volumes, and timbral characteristics of four different voices. The combination of controlling note volumes and channel volumes in parallel provided the possibility for extreme crescendo/diminuendo effects, which was fun for the performer but also required greater practice and concentration. At the end of the piece there is a rhapsodic conclusion which is also under the performer’s direct mix control. The composition is an adaptation of an alaap written by my teacher, Pandit Sreeram Devasthali, in 1993, and is set in the North Indian raag called Maarwa, which is highly chromatic.