The SmartDesk is a desktop and large visual display combined with a wide baseline stereo vision system, and other perceptual apparatus.
The stereo vision system tracks the user's head and hands in 3D at 30 fps. It utilizes a physical model of the human body to extrapolate untracked body parts as well as to interpolate through stereo estimation errors. Probabilistic models of user behavior are built on top of this system.
There is also a whole room version of this system