The Seagull: A Performance Animation System

The Seagull is a performance animation system based on the Smart Desk computer vision system developed at the Vision and Modeling Group at the MIT Media Lab. In a nut shell, the system uses two cameras to recover the 3-D position of the head and hands of the user in real time. The idea of the Seagull is to let the user configure the mapping from the head and hands to the configuration of the character on the screen. This is accomplished during a short training phase in which the user is instructed to match the position of the bird on the screen - once with the seagull with wings down and another when the wings are up. When the training is completed, the user can then "flap" his or her wings. The faster the flapping, the faster the landscape scrolls by. If the user stops flapping, the seagull slows down to soar over the landscape. Note that because the user specifies the correspondence between his or her body and the bird, the user can "flap" in essentially any way desired. The landscape images were taken with permission from KCTV's production Over New England, which includes beautiful aerial photography of some famous New England landmarks. Other touches include the control of the bird's banking and position in depth by using the position of the user's head, and automatic sensing of when the user's head bobs up and down quickly, which triggers an authentic seagull mating call (a real crowd pleaser). Stereo wind sound surrounds the user. The Seagull was presented at the 1996 Siggraph Digital Bayou as a part of the MIT Media Lab's Smart Spaces booth. Luxomatic is another vision-based performance animation system I've put together. (7.0MB/no audio) Here's a QuickTime 4 video of the Seagull. Technical Notes The landscape video sequences were uncompressed from disk on the fly from a Digital Alpha workstation equipped with an AV321 video card. The 3-D model of the seagull was donated by Viewpoint DataLabs. Softimage 3D was used to modify the model to allow articulation of the wings. The texture map was drawn in pencil and scanned. The resulting Open Inventor model was rendered and chroma-key composited onto the landscape video in real time by an SGI Indigo workstation. The Smart Desk system uses two SGI Indy workstations, one for each camera.