Perception: Deictic Gestures

Deictics are the only form of gesture supported by Ogg That There. They are labeled by speech events, not actually recognized. Interpretation of deictics relies on interpolation over a set of calibration examples obtained off-line by asking the user to point at the four corners of the screen with both hands in turn. This results in four sets of measurements for each hand. Separate calibrations are maintained for each hand.

In general these four points will not form parallelograms in feature space, so linear strategies introduce unacceptable warping of the output space. Ogg That There employs a perspective warping to translate input features ((x,y) hand position in 3D space) to screen position:

$$\left[ \begin{array}{c} XW \\ YW \\ W \end{array} \r... ...\left[ \begin{array}{c} x \\ y \\ 1 \end{array} \right]$$ (3.1)

where X and Y are screen position and x and y are hand position. The parameters a,b,c,d,e,f,g,h are estimated from data. With some manipulation[25] the above equation can be rewritten as a linear system of equations:

$$\left[ \begin{array}{cccccccc} x_1 & y_1 & 1 & 0 & 0 & 0 ... ...\ Y_2 \\ \vdots \\ X_n \\ Y_n \\ \end{array} \right]$$ (3.2)

Interpolation of a novel deictic simply involves plugging the new (x,y) into Equation 3.1. The resulting screen coordinates, (X,Y), are then passed to the Display which does a pick action to convert those coordinates into a game object.