The Inverse Observation Model

In the open-loop system, the vision system uses a Maximum Likelihood (ML) framework to label individual pixels in the scene:

$$\Gamma_{ij} = \arg \max_{k} \left[ \Pr( {\bf O}_{i j} \vert {\mbox{\boldmath$ \mu $ }}_k, {\bf K}_k ) \right]$$ (10)

where $\Gamma_{ij}$ is the labeling of pixel (i,j), and $({\mbox{\boldmath$\space \mu $ }}_k, {\bf K}_k)$ are the second-order statistics of model k.

To close the loop, we need to incorporate information from the 3-D model. Given the current state of the model ${\bf q}$, it is possible to compute the state of an individual link that matches a specific tracked feature (say the hand), and call it ${\bf v}$. Then, given a model of the camera, it is possible to calculate the perspective projection of that state into 2-D and call it ${\bf v}^{*}$.

Since the vision system uses a stochastic framework, it is necessary to represent this link projection as a statistical model: $\Pr( {\bf O}_{i j} \vert {\bf v}_k^{*} )$. Integrating this information into the 2-D statistical decision framework results in a Maximum A Posteriori decision rule:

$$\Gamma_{ij} = \arg \max_{k} \left[ \Pr ({\bf O}_{i j} \vert {... ...f K}_k) \cdot \Pr( {\bf O}_{i j} \vert {\bf v}_k^{*} ) \right]$$ (11)