SmartDesk Home Page

SmartDesk is a project of the Perceptual Computing group at the MIT Media Lab and encompasses experimentation on a range of computer-based perceptual input and output systems in a personal work environment.

Hardware

The wide-baseline stereo is used for visually tracking the macroscopic movements of the user. The foveating (pan-tilt-zoom) camera is used to obtain high-resolution images of an area of interest, based on various attention-focusing algorithms. The phased-array microphone is used to pick up audio from a direction of interest, usually from the user's head. And the loudspeakers are used to generate transaural spatial audio.

Applications

• Planet: visually-guided interaction and transaural rendering
• Puppet: visually-guided animation
• Visually-guided face recognition

Movie

• 34MB movie

Self calibration

• Ali's self-calibration page

3-D person tracking

• Ali's person-tracking page
• Chris's pfinder page
• Irfan's head tracking page

Face recognition

• Baback's face recognition home page

Facial expression analysis

• Irfan and Sumit's work on viewing faces
• Irfan's facial motion page
• Irfan's facial expression page

Gesture, action, and event analysis

• Irfan's page on causal analysis of gesture

American Sign Language

• Thad's ASL page

Vision-steered audio input

• Mike's mic page

Transaural audio rendering

• Mike's mic page

Self calibration

• A. Azarbayejani and A. Pentland, Real-time self-calibrated stereo person tracking using 3-D shape estimation from blob features, TR#363, January 1996 (Submitted to ICPR'96)

3-D person tracking

• A. Azarbayejani and A. Pentland, Real-time self-calibrated stereo person tracking using 3-D shape estimation from blob features, TR#363, January 1996 (Submitted to ICPR'96)
• Sumit Basu, Irfan Essa, and Alex Pentland, Motion Regularization for Model-based Head Tracking
• Christopher R. Wren, Ali Azarbayejani, Trevor Darrell, Alex Pentland (1995), Pfinder: Real-Time Tracking of the Human Body Appears in: SPIE Photonics East 1995, Vol. 2615 pp. 89-98

Face recognition

• Baback Moghaddam, Chahab Nastar and Alex Pentland (1996), Bayesian Face Recognition using Deformable Intensity Surfaces Appears in: Proceedings of IEEE Conference on Computer Vision & Pattern Recognition, San Francisco, June 1996.
• Chahab Nastar, Baback Moghaddam and Alex Pentland (1996), Generalized Image Matching: Statistical Learning of Physically-Based Deformations

Facial expression analysis

• Irfan Essa, Sumit Basu, Trevor Darrell, Alex Pentland (1996), Modeling, Tracking and Interactive Animation of Faces and Heads using Input from Video Appears in: Proceedings of Computer Animation '96 Conference, Geneva, Switzerland, June 1996. IEEE Computer Society Press
• Irfan Essa and Sandy Pentland (1995), Coding, Analysis, Interpretation, and Recognition of Facial Expressions

Gesture, action, and event analysis

• A. Pentland (1995), Machine Understanding of Human Action
• A. Wilson and A. Bobick (1995), Learning Visual Behavior for Gesture Analysis
• L. W. Campbell and A. Bobick (1995), Recognition of Human Body Motion Using Phase Space Constraints Appears in: Proc. of ICCV 95, Pp. 624-30.

American Sign Language

• Thad Starner (1995), Visual Recognition of American Sign Language Using Hidden Markov Models (S.M. Thesis)

Vision-steered audio input

• M. Casey, W. Gardner, and S. Basu, Vision steered beam-forming and transaural rendering for the Artificial Life Interactive Video Environment, TR#352

Transaural audio rendering

• M. Casey, W. Gardner, and S. Basu, Vision steered beam-forming and transaural rendering for the Artificial Life Interactive Video Environment, TR#352