Mapping 2D Feature Tracking into the Kalman Filter

As was discussed previously, each feature tracker recovers an optimal $\bf {\mu}$ motion parameter by minimizing $O(\bf {\mu}\rm )$. However, since the 2D feature tracking in question was being used to recover translation, rotation and scale, the $\bf {\mu}$ vector has 4 degrees of freedom (not merely 2). We can represent these 4 degrees of freedom as 2 point features that are free to translate independently. In other words, two arbitrary points on the correlation window are selected (i.e. 2 opposing corners) and it is trivial to compute their locations from a corresponding $\bf {\mu}$ transformation (translation, scale and rotation). This mapping goes both ways and we can model the 2D tracking for each image patch with the SSD model using $\bf {\mu}$ or using the positions of 2 distinct feature points somewhere within the window (X₁, Y₁, X₂, Y₂). For M correlation-based windows, we compute the (X,Y) location of N=2M points. These feature points are then arranged into the $\bf {y}$ vector for input into the EKF.