TR#350: Machine Understanding of Human Action

Alex Pentland

Short version available in:
7th Int'l Forum on Frontier of Telecom Technology
Tokyo, Japan

The Perceptual Computing Section of the Media Laboratory is working to make computers that understand people, and can work with them in the manner of an attentive human-like assistant. To this end we have built a series of interactive office spaces that are used as real-time experimental testbeds. These spaces are instrumented with cameras and microphones, and perform audio-visual interpretation of human users. Real-time capabilities include 3-D tracking of head, hands, and feet; ``holographic'' audio; face recognition; and interpretation of face and hand gestures. People in the space can control programs, browse multimedia information, and experience shared virtual environments without wires or special goggles.