Hidden Markov Models of Control

Since human motion evolves over time, in a complex way, it is advantageous to explicitly model temporal dependence and internal states in the control process. A Hidden Markov Model (HMM) is one way to do this, and has been shown to perform quite well recognizing human motion[21].

The probability that the model is in a certain state, S_j given a sequence of observations, ${\bf O}_{1}, {\bf O}_{2}, \ldots, {\bf O}_{N}$, is defined recursively. For two observations, the density is:

$$\Pr( {\bf O}_{1}, {\bf O}_{2}, {\bf q}_2 = S_j ) = \left[... ...\pi_{i} b_{i}({\bf O}_1){\bf a}_{ij} \right] b_{j}({\bf O}_2)$$ (2.12)

Where $\pi_{i}$ is the prior probability of being in a state i, and $b_{i} ({\bf O})$ is the probability of making the observation ${\bf O}$ while in state i. This is the Forward algorithm for HMM models.

Estimation of the control signal proceeds by identifying the most likely state given the current observation and the last state, and then using the observation density of that state as described above. We restrict the observation densities to be either a Gaussian or a mixture of Gaussians. There are well understood techniques for estimating the parameters of the HMM from data.