A Model for Control

Kalman filtering includes the concept of an innovations process. This is the difference between the actual observation and the predicted observation transformed by the Kalman gain:

$${\mbox{\boldmath$ \nu $}}_t = {\bf K}_t ({\bf y}_t - {\bf H}_t{\mbox{\boldmath$ \Phi $}}_t\hat{\bf x}_{t-1})$$ (2.10)

The innovations process ${\mbox{\boldmath$\ \nu $}}$ is the sequence of information in the observations that was not adequately predicted by the model. If we have a sufficient model of the observed dynamic process, and white, zero-mean Gaussian noise is added to the system, either in observation or in the real dynamic system itself, then the innovations process will be white. Inadequate models will cause correlations in the innovations process.

Since purposeful human motion is not well modeled by passive physics, we should expect significant structure in the innovations process.

A simple example is helpful for illustrating this idea. If we track the hand moving in a circular motion, then we have a sequence of observations of hand position. This sequence is the result of a physical thing being measured by a noisy observation process. Assuming that the hand moves according to a linear, constant velocity dynamic model, it is possible to estimate the true state of the hand, and predict future states and observations. If this model is sufficient, then the errors in the predictions should be solely due to the noise in the system.

The upper plots in Figure 2.3 show that model is not sufficient. Smoothing ${\mbox{\boldmath$\ \nu $}}$ reveals this significant structure (top left). Plotting the innovations along the path of observations make the relationship between the observations and the innovations clear: there is some un-modeled process acting to keep the hand moving in a circular motion (top right). This un-modeled process is the purposeful control signal that being applied to the hand by the muscles.

In this example, there is one active, cyclo-stationary control behavior, and it's relationship to the state of the physical system is straightforward. There is a one-to-one mapping between the state and the phase offset into the cyclic control, and a one-to-one mapping between the offset and the control to be applied. If we use the smoothed innovations as our model and assume a linear control model of identity, then the linear prediction becomes:

$$\hat{\bf x}_{t} = {\mbox{\boldmath$ \Phi $}}_t\hat{\bf x}_{t-1} + {\bf I}{\bf u}_{t-1}$$ (2.11)

where ${\bf u}_{t-1}$ is the control signal applied to the system. The lower plots in Figure 2.3 show the result of modeling the hand motion with a model of passive physics and a model of the active control. The smoothed innovations are basically zero: there is no part of the signal that deviates from our model except for the observation noise.

In this simple, linear example the system state, and thus the innovations, are represented the same coordinate system as the observations. With more complex dynamic and observations models, such as described in Section 2.2.1, they could be represented in any arbitrary system, including spaces related to observation space in non-linear ways, for example as joint angles.

The next section examines a more powerful form of model for control.