Results

**Figure 2:** Frames on the left show video and 2-D blobs from one camera in the stereo pair. Frames on the right show corresponding configurations of the dynamic model at that instant in time.
$\begin{figure} \centerline{\psfig{figure=figs/skel10.ps,width=90mm}}\centerline{\psfig{figure=figs/skel08.ps,width=90mm}}\end{figure}$

The dynamic skeleton model currently includes the upper body and arms. Figure 2 shows the real-time response to various target postures. The model interpolates those portions of the body state that are not measured directly, such as the upper body and elbow orientation, by use of the model's intrinsic dynamics and the behavior (control) models. The model also rejects noise that is inconsistent with the dynamic model. Table 1 compares RMS noise in the dynamic model output with noise in the underlying feature tracker. The ``line following'' test measures error from the best-fit line to data produced by constraining the users hand to move along a linear trajectory. The ``rotational jitter'' measures error to a smoothed version of data obtained by smooth motions of the user's hand through a rotation.

Table 1: Comparison of RMS tracking error for tracking with and without dynamic feedback.

	tracker		dynamic model
line following	1.4	cm	0.9	cm
rotational jitter	2.2	deg	0.6	deg

**Figure:** Tracking performance on a sequence with significant occlusion. **Top:** A diagram of the sequence and a single camera's view of the motion **Left:** This graph shows an example of tracking results without feedback. **Right:** This shows an example of correct tracking when feedback is enabled. Notice the smooth traces in the YT and ZT projections.
$\begin{figure} \centerline{\psfig{figure=figs/rumbaD.ps,width=26mm} \psfig{fig... ...e{2mm} \centerline{\psfig{figure=figs/helix_shadow.ps,width=130mm}}\end{figure}$

It can be seen that Figure 3 illustrates another advantage of feedback from higher-level models to the low-level vision system. Without feedback, the 2-D tracker fails if there is even partial self-occlusion from a single camera's perspective. With feedback, information from the dynamic model can be used to resolve ambiguity during 2-D tracking.

**Figure 4:** Users sharing the workspace. Physical constraints stabilize the 2-D tracker with respect to competing targets.
$\begin{figure} \centerline{\psfig{figure=figs/multi.ps,width=90mm}}\end{figure}$

The model predictions also stabilize tracking by providing constraints that help the tracking algorithm reject distractions in the environment. The addition of another person to the scene, as in Figure 4, produces many patches in the image that are similar to the target blobs. Without high-level model knowledge, the 2-D tracker can only reject these distractions based on some assumptions about the temporal stability of blobs. With the addition of high-level feedback, however, the 2-D tracker now has information about the physical constraints of the underlying system. Consequently, it is generally not distracted by competing targets (such as other people).

The full system currently runs at at 30Hz utilizing four computers: 60% of an SGI 175MHz R10K O₂ per camera for blob-level processing, 100% of an Alpha 500MHz 21164 for dynamics and constraints, and 30% of an SGI 195MHz R10K Indigo² Impact for stereo processing and graphics rendering. The computers are connected to 100Mbps switched ethernet. The optimizations described in Section 2.3 reduce the utilization of the Alpha to 15%. That optimized dynamics and constraint module could be migrated to one of the under-utilized SGIs.