%\documentstyle[psfig,epsf,twocolumn]{article}
\documentstyle[cup7b,david,epsf]{cuphci}


%FROM M. BLACK ICCV95
\def\x{\mbox{\bf x\/}}

\def\X{\mbox{\bf X}}
\def\A{\mbox{\bf A}}
\def\P{\mbox{\bf P}}
\def\C{\mbox{\bf C}}
\def\x{\mbox{\bf x}}
\def\z{\mbox{\bf z}}
\def\l{\mbox{\bf l}}
\def\m{\mbox{\bf m}}
\def\bi{\mbox{\bf i}}
\def\u{\mbox{\bf u}}
\def\a{\mbox{\bf a}}
\def\p{\mbox{\bf p}}
\def\d{\mbox{\bf d}}
\def\Q{\mbox{\bf Q}}
\def\s{\mbox{\bf s}}
\def\R{{\cal R}}
\def\calD{{\cal D}}
\def\calS{{\cal S}}
\def\g{\mbox{\bf g}}
\def\flow{\{\mbox{\bf u}\}}

% up = u-predicted (u^-)
% ue = u-estimated (u^+)

\def\up{\mbox{\bf u}^-}
%\def\ue{\mbox{\bf u}^+}
\def\ue{\mbox{\bf u}}
\def\ip{i^-}
%\def\ie{i^+}
\def\ie{i}
\def\lp{l^-}
%\def\le{l^+}
\def\le{l}

\def\G{{\cal G}}
%\def\C{{\cal C}}
\def\M{{\cal M}}
\def\I{{\cal I}}
\def\L{{\cal L}}

\def\grad{\nabla}

\input{psfig-macro}


\begin{document}
%don't want date printed
\date{}

%make title bold and 14 pt font (Latex default is non-bold, 16 pt)
%\title{\Large\bf Looking at People in Action-- An Overview}

%for single author (just remove % characters)  
\author[Y.\ Yacoob et al. ]{Y.\ Yacoob,  L.\ Davis, M.\ Black${}^*$, D.\ Gavrila, T.\ Horprasert and C.\ 
Morimoto \\   %%%% check that this format is the same for all other chapters
  Computer Vision Laboratory \\
  University of Maryland \\
  College Park, MD~~20742}
\chapter{ Looking at People in Action -- An Overview}

%\maketitle
\renewcommand{\thefootnote}{}
\footnotetext{The Support of the Defense Advanced Research Projects
Agency (DARPA Order No.\ C635) under Contract N00014-95-1-0521 is gratefully acknowledged.\\
$*$ with Xerox PARC}

%I don't know why I have to reset thispagestyle, but otherwise get page numbers
%\thispagestyle{empty}

\section{Introduction}

\renewcommand{\thefootnote}{*}
In this paper we overview current and recent research activity in our group.
Section \ref{FEsection} provides a review of the research on recognition of facial
expressions. Section \ref{HGsection} discusses two approaches to recognition
of head gestures. In Section \ref{HPOSE} we discuss an approach for head orientation 
estimation from a single image. 
Section \ref{DARIU} provides a model-based approach for tracking and recognition
of human body actions. 

\input{FEintro}
\input{FEmodels}
\input{FEresults}
\input{FEnew-mid-high}


\section{Head Gesture Recognition}
\label{HGsection}
The planar face motion  model can also be used
to recognize head gestures. We developed a rule-based system and a Hidden Markov
Model system for this purpose.

\input{florida}


\subsection{Hidden Markov Models}

The block diagram of the HMM  head gesture recognition system is given
in Figure \ref{fig:BlockDiag}.
The vector quantization
module
transforms the motion parameters into observation sequences that
are used by the HMMs. The HMMs, after training,
score the input observation sequence, and the model
with the highest score (i.e., maximum likelihood) determines the
output symbol. 

\begin{figure} [htb]
  \centerline
  {
     \psfig{figure=chapter_figs/BlockDiag.ps,width=3.0in}
  }
  \caption{Block diagram of the head gesture recognition system}
  \label{fig:BlockDiag}
\end{figure}



We use a dynamic mechanism for
selecting an appropriate threshold before the quantization process. First, a
low threshold is used to localize gestures in the input sequence. It is
assumed that periods of silence (i.e., all image rotations close to zero)
separate the presentation of a gesture. After the localization of a gesture
in the sequence, the energy of each parameter is computed.
High energy signals receive low thresholds and low energy signals receive
high thresholds to reduce the effect of spurious noise.


Each of these image
rotations can be considered to be positive $(+)$, negative $(-)$, or
zero $(0)$.
With three independent parameters which can assume three different values, 27 symbols
are necessary to code a single state (e.g., the symbol ``0'' can be used to
code the parameter state (+, +, +), the symbol ``1'' to code (+,+,$-$), and
so on, until all combinations are coded).

To facilitate the training and recognition operation modes of the HMMs,
we further reduce the number of observation symbols by mapping the image
rotation parameters into the seven
symbols: UP, DOWN, LEFT, RIGHT, CLOCKWISE, ANTICLOCKWISE, and REST. A symbol is 
selected as
output based on the dominant motion which is determined by the signal with
highest energy. UP or DOWN are selected when image-pitch is dominant,
LEFT or RIGHT when image-yaw is dominant, and CLOCKWISE or ANTICLOCKWISE when image-
curl is
dominant. When none of the signals is dominant, the symbol REST is selected.


The image rotations are thresholded and mapped to the seven
possible observation symbols, depending on the instantaneously
highest energy band.
Although the system could work by directly considering the overall highest
energy band, we choose this approach of selecting the instantaneous
highest band to be able to model more complex gestures. 

For our experiments we developed one
HMM for each of nod (YES), negate (NO), approximate (MAYBE) and
greet (HELLO).
The models were trained using 28 sequences taken from five people.
The test data consisted of 33 sequences taken from six 
other people. 
Due to the simplicity of the models, the parameters of each model were
initially ``guessed'' and then tunned by the Baum-Welch iterative process. 

Figure \ref{Yas-fig:TableConfusionMatricesTest} shows the results for the
trained HMMs using the test data. Observe that the HELLO
gesture is confused most of the time with the YES gesture. This is caused by
the definition of the HELLO gesture, that is considered to be a single
nod. Looking at the input sequence data for the HELLO gesture,
most subjects present more than a single nod,
at least 1.5 cycles which characterizes the YES gesture. The YES gesture
is also confused with the HELLO gesture because some subjects when told
to perform the YES gesture, executed only one image-pitch cycle.
Changing the definitions
in order to allow the HELLO gesture to have
up to 2 cycles and considering the YES gestures to be those sequences with
more the 2 image-pitch cycles
would solve this problem for this particular data set.

\begin{table} [htb]
\centerline {
\scriptsize
\begin{tabular} {|l|c|c|c|c|} \hline
  & YES & NO & MAYBE & HELLO  \\ \hline
  YES   & 13 &   & 2 & 3      \\ \hline
  NO    &    & 5 &   &        \\ \hline
  MAYBE &  1 & 1 & 4 &        \\ \hline
  HELLO &  2 &   &   & 2      \\ \hline
  TOTAL & 16 & 6 & 6 & 5      \\ \hline
\end{tabular}
}
\caption{Confusion matrix of test data for trained HMMs}\label{Yas-fig:TableConfusionMatricesTest}
\end{table}



\section{Head Orientation Estimation} \label{HPOSE}

\input{TH.int}
\input{TH.HPcomp}
%\input{TH.err_analysis}
\input{TH.exp}

\input{dariu}
\bibliography{yaser}

\end{document}


