Smart Rooms

Smart Rooms act like invisible butlers. They have cameras, microphones, and other sensors, and use these inputs to try to interpret what people are doing in order to help them. We have already built smart rooms that can recognize who is in the room and can interpret their hand gestures, and smart car interiors that know when drivers are trying to turn, stop, pass, etc., without being told.

Smart Rooms share many philosophical similarities with our Smart Desks and Smart Clothes projects.

Application Environments

The Smart Room can provide an unencumbered user-interface to a virtual environment. It utilizes vision-based tracking built by Christopher R. Wren, Trevor Darrell , Ali J. Azarbayejani, (and others) and a large projection screen to eliminate the need for invasive sensors, head mounted displays and the annoying umbilicus of wires that come with them.

Information Spaces: City of News is an immersive 3D web browser City of News is a dynamically growing urban landscape of information. It is an immersive, interactive, web browser that takes advantage of people's strength remembering the surrounding three-dimensional spatial layout. Starting from a chosen "home page", where home is associated with a physical space, our browser fetches and displays URLs so as to form skyscrapers and alleys of text and images through which the user can "fly".
Real-Time gesture and speech recognition allow natural interaction with information shown on a large screen.
Virtual Studio: Digital Circus: Digital Circus is a 3D virtual circus in which people can meet and interact among themselves or with circus performers. As in a magic mirror , advanced computer vision techniques allow participants to see their full body video image composited in 3D space - without the need for a blue screen background . Hence such setup can be used at home for collaborative storytelling, visual communication from remote locations, or game playing. The participant's image is subjected to all graphics transformations that can apply to graphical objects, including scaling. According to the participant's position in the space, his/her image occlude or are occluded by virtual objects in respect to the 3D perspective of the virtual scene. Multiple people can connect from remote locations, therefore turning the magic mirror into a magic space .
ALIVE: The Aritificial Life Interactive Video Environment (ALIVE) uses a magic mirror idiom to create an enhanced world for the user on the other side of the looking glass. The IVE environment allows the user to remain unencumbered, and therefore increases the immersive aspect of the interface. The use of autonomous agents allows the enhanced world to be believable. Video compositing is used to place the user and agents into a single virtual space.
ALIVE is a joint project between the the Vision and Modeling Group and the Autonomous Agents Group at the MIT Media Lab.
Telepresence: For SigGraph '95, the idiom was extended to allow two (or more) ALIVE spaces to share common alternate realities. Instead of just seeing video of themselves, users are now represented, locally and remotely, by avatars: sometimes human, sometimes realistic video, often not. Since the virtual world contains model-based representations of the agents and the users, the communication bandwidth between stations is very low: much less than 56Kbps. We have successfully shared virtual spaces not only within the building, but also across the continent (Boston to LA for Siggraph 95), and even across the World (Boston to London (BT) for the Media Lab's 10th Anniversary).
Performance Spaces: Flavia Sparacino is developing systems that explore the usefulness of Smart Rooms in artistic and expressive applications including dance and theatrical performances.
American Sign Language: Machine Vision Recognition of American Sign Language Using Hidden Markov Models demonstrates a high accuracy (up to 99.2%), automatic system for recognizing ASL in the Smart Room.
SmartCars: Work at Nissan Cambridge Basic Research Lab has extended the idea of the Smart Room to include Smart Cars . Automobile drivers' intended action (e.g., to turn, change lanes, brake, etc.) can be inferred by observing their control inputs (steering and acceleration) as they prepare to execute the action. Actions are modeled as a sequence of internal mental states, each with a characteristic pattern of driver control behavior; this is similar to the hidden Markov modeling used in speech recognition. By observing the temporal pattern of the drivers' control inputs and comparing to the action models, we can determine which action the drivers are beginning to execute.
Hyperplex: Flavia Sparacino and others are working on a system that uses IVE to navigate virtual spaces populated by not only information, but information that reacts to you and your interests (driven by the same technology that powers Silas).
SURVIVE: Ken Russell and others used the IVE system to create an interface to the game Doom . Called SURVIVE (for Simulated Urban Recreational Violence IVE), it provides a much more compelling interface than the standard "bang on the keyboard" approach.

References

Contributors

A list of contributors is available for the SigGraph '95 demo.

Back up to the Demos menu
Wed Oct 18 09:01:12 1995