Mathematical Framework

  

Figure: The flow of information though the system. 2-D observations are combined to drive a 3-D dynamic model of the skeleton. This dynamic model uses a control law that chooses `typical' behaviors when it is necessary to choose among multiple possible legal trajectories. Predictive feedback from the dynamic model is provided by setting priors for the 2-D observations process.

The human body is a complex dynamic system, whose visual features are time-varying, noisy signals. Accurately tracking the state of such a system requires use of a recursive estimation framework, as illustrated in figure 1. The elements of the framework are the observation model relating noisy low-level features to the higher-level skeletal model and vice versa, the dynamic skeletal model, and a model of typical behaviors. Section 4 explains how, with a bit of work, it is possible to approximately model the physical aspects of the human body. Section 5 explains the observation model. The next section explains an approach for modeling human behaviors.