INTRO:
-----
The broad goal: The goal of the face recognition software is to take
an image that contains a person's face, to project the image onto a set
of eigenfaces and produce a set of coefficients, to record the
coefficients for that image, and to perform recognition by comparing
the coefficients from one image to those from others.
For information on the directory structure please read DIRINFO.
BEGINNING THE LOOP:
------------------
You have an image on disk and you want to process it to produce
its set of coefficients. FaceRecInterface contains the necessary code
to take an image and compute the eigencoefficients for recognition.
/*---------------------------------------------------------------------------
Program Name: faceRecInterface.cc
Purpose: The idea of the software is to get an image, normalize it,
and project it onto an eigenface basis in order to compute
a description of the image.
Author: Tanzeem Choudhury
Baback Moghaddam
-----------------------------------------------------------------------------*/
/*---------------------------------------------------------------------------
Function Name: loadEigen()
Purpose: function for loading eigenvectors into memory.
Author: Tanzeem Choudhury, Baback Moghaddam
---------------------------------------------------------------------------*/
This function loads the image to memory. Bytes per pixel identifies
whether the image is raw chars (bytesPixel =1) or raw floats
(btyesPixel=4). Alll the images provided are raw chars.
/*---------------------------------------------------------------------------
Function Name: loadKnownFaces()
Purpose: function for loading coefficients of known faces
into memory.
Author: Tanzeem Choudhury
---------------------------------------------------------------------------*/
This function simply loads the eigencoefficients of known faces off
the disk from the directory knwon_dir and returns them in the data
structure, kfaces.
/*---------------------------------------------------------------------------
Function Name: loadEigen()
Purpose: function for loading eigenvectors into memory.
Author: Tanzeem Choudhury, Baback Moghaddam
---------------------------------------------------------------------------*/
This function simply loads the eigenvectors off the disk from the
directory eigdir and returns them in the data structure, eigimages.
Neigframe_in is the number of eigenimages to load.
/*---------------------------------------------------------------------------
Function Name: loadTemplates()
Purpose: function for loading BF format files into memory
Author: Tanzeem Choudhury
---------------------------------------------------------------------------*/
This routine loads data from BF format files: dmf%dt%.bf from the
datapath and a feature model matrix (in BF) format as well as a
geometry_musigma.bf, model.bf and ,ask.bf files
/*----------------------------------------------------------------------
FUNCTION: scalewarp()
AUTHOR: Dave Becker
This function, scalewarp, scales and translates an image so as to
place the (x,y) point at (x_dest,y_dest).
------------------------------------------------------------------------*/
The image is now scaled and translated so that the head is in the
proper place. However, the face might be tilted or not perfectly
aligned.
/*----------------------------------------------------------------------
FUNCTION: eigsearch()
AUTHOR: Baback Moghaddam, Tanzeem Choudhury
------------------------------------------------------------------------
Local feature search by eigentemplates
This routine uses data read in earlier from the following files
stored in datapath:
1. features.bf which is an N-by-2 matrix defining the size (row,col)
of each of the N templates.
2. template[n].bf where n=1...N is the eigenvector ROW matrix where
the 1st row is the mean feature, and the remaining rows
are the principal eigenvectors 1:M.
NOTE: The eigenvectors are stored in column-order (as in MATLAB)
3. variances.bf which is an N-by-M matrix, where each row represents
the rank-ordered eigenvalues associated with that feature
4. windows.bf
5. umap_musigma[n].bf
6. spatial_musigma[n].bf
----------------------------------------------------------------------
NOTE: In this version if unitmap'ing is selected it will use
the Mahalanobis mean/stddev in addition to the standard
distance computation.
------------------------------------------------------------------------*/
Eigsearch() is picking out _candidate_ feature locations, rather than
picking _the_ feature locations (again, see main.ps for more detail).
These candidate feature locations are returned to ml_detect(), which
picks from between these candidates fthe most likely locations of
features.
/*----------------------------------------------------------------------
FUNCTION: ml_detect()
AUTHOR: Baback Moghaddam, Tanzeem Choudhury
------------------------------------------------------------------------
Feature Geometry Validation using Maximum Likelihood Minimization
This routine uses data from BF format files: dmf%dt%.bf from the templates
and a feature model matrix (in BF) format as well as a geometry_musigma.bf
file
And dumps the coods of best feature matches in output data structure
---------------------------------------------------------------------- */
Ml_detect(), basically, compares the geometry of candidate feature
locations to the rough geometry that it is known features should lie
in, and gives a score corresponding to how well the features match
that geometry.
/*--------------------EIG -> Load eigentemplate data ------------------*/
variance, unit map, and spatial information (see main.ps).
/*--------------------EIG -> windows stuff -----------------------*/
The window.bf file controls how big of an area eigsearch() searches in
when looking for features. In other words, eigsearch is handed a head
location. It knows that the left eye is sort of off to the left and
above the head center, so rather than search the whole image for the
left eye, eigsearch() just wants to search in an area around there.
This windows.bf file contains the size of the window to use for that search.
Now that the feature locations are known, the image is warped again so
that the features are perfectly normalized. That is, the eyes are set
to be in given locations (the interocular distance is normalized, that
is), and the rest of the image is tilted so that the mouths all lie
along the same line (the mouth is set to be directly below the
midpoint of the two eye locations). This is taken care of in
modelwarp_mc() :
/*----------------------------------------------------------------------
FUNCTION: modelwarp_mc()
AUTHOR: Baback Moghaddam, Tanzeem Choudhury
------------------------------------------------------------------------
This contains code to warp images according to the affine transform
between the feature locations indicated and those in
the model data structure (from the model.bf file)
---------------------------------------------------------------------- */
The final bit of processing that occurs is that a mask is
applied to the image. At this point, the image has been warped so
that the face lies in a particular place, with the eyes and mouth also
so normalized. The masking is a step to get rid of the effect of the
background and of hair. So, another image is simply dot-multiplied to
the image being processed. The image (the mask) being multiplied by
our image is set to be one where faces are and zero where the
background is, with a little bit of smoothing in between. The two
images are multiplied, with the result being an image that has a face
in the middle and black around the edges.
The code to the do the masking is included in modelwarp_mc().
The processed image is now ready to be projected onto the
eigenvectors, so the image is sent to the function computeCoeffs():
/*---------------------------------------------------------------------------
FUNCTION: computeCoeffs()
Purpose: Given an input image eigenvectors, produces
coefficients by projection.
-----------------------------------------------------------------------------*/
This function simply takes the processed image, and sees how much of
each eigenface it is made up of (see main.ps for the details).
Once these coefficients have been computed, there are two things that
can be done. Either the image just processed is meant to be
enrolled into the database, or the image just processed is meant to be
compared to people already enrolled in the database and recognized.
/*---------------------------------------------------------------------------
FUNCTION: writeCoeffs()
Purpose: Writes the coefficients for an image into a file
-----------------------------------------------------------------------------*/
This procedure simply takes the coefficients and writes them out into
a file with the same name as the image filename in the directory of
known faces (the database).
If the image was not meant to be enrolled, but recognized instead,
then neighbor() is called instead of writeCoeffs().
/*---------------------------------------------------------------------------
FUNCTION: neighbor()
Purpose: function for taking coefficients and
figuring out which among known faces has closest set
of coefficients.
Author: Dave Becker
Wasiuddin Wahid
Tanzeem Choudhury
-----------------------------------------------------------------------------*/
Neighbor() takes the set of known faces (this is loaded earlier via
loadKnownFaces()) and the new set of coefficients and compares them.
It sees which known face has coefficients most similar to the newly
computed coefficients. It does this basically by finding the
Euclidean distance between two sets of coefficients via
find_distance() and then maintaining a linked-list stack of faces that
is ordered in terms of distance (this is done via add_match(), which
simply inserts a new face into the stack depending on the distance.
firstCoeff and lastCOeff simply defines the coefficients to use.
/*---------------------------------------------------------------------------
FUNCTION: print_match()
Purpose: function for printing the n closest match
Author: Dave Becker
Wasiuddin Wahid
Tanzeem Choudhury
This function prints out the first n matches to the new set of
coefficients and dumps it out to a file.
-----------------------------------------------------------------------------*/
Eigsearch(), and ml_detect() are assuming 128 by 128 image. The
maskfile included with this version of the software is 128 by 128.
DO_RIGID is used the second time the image gets scalewarped. That is,
after the features have been located, the image is warped so that the
eyes are aligned and the mouth is in the right place. This can either
be done with a rigid (rotation and translation) warp (fixing
DO_RIGID_PT1 and DO_RIGID_PT_2 in particular locations) or with an
affine warp. DO_RIGID sets which to do: 1 for rigid, 0 for affine.
X_DEST and Y_DEST is the x,y coordinates where you would like the head
center to be located after the image is fully normalized. The current
numbers, 59 and 72, are a function of the size of the image. If you
change image size, you should change X_DEST and Y_DEST accordingly.
=========================================================================
/*---------------------------------------------------------------------------
Program Name: klt.c
Purpose: Performs the Discrete Karhunen-Loeve Transform to compute
the set of eigenimages for a set of images.
Author: Baback Moghaddam
-----------------------------------------------------------------------------*/
This is the code for calculating eignevector given training images. If
you need to recalculate the eigenvectors, this is the code to use. The
input images should be training normalized training images. All the
preprocessing for computing coefficients and computing eigenvectors is
the same. You probably don't need use this right now. There are some
functions missing, I can either send it to you or put it in the
directory shortly. I should mention that the eigenvectors were
calculated from a large set of face images and probably will be the
best ones to use.
That's it!
I hope this clears things up for you, but do please feel free to
contact me with any questions.
Tanzeem Choudhury
tanzeem@media.mit.edu