Face Detection & Recognition

Dr. Michael S. Lew, Dr. Nicu Sebe, Dr. Nies Huijsmans, Roel Hoogenboom

{mlew,nicu,huijsman,rhoogenb}@wi.leidenuniv.nl

Project Goal

The goal of this ongoing project is to formulate paradigms for detection and recognition of human faces in complex backgrounds. One of the applications would be towards adding face oriented queries to our image database project. For instance, "Find all the images with 3 or more faces." With recognition capabilities, we could then augment the queries to identity oriented retrieval. For example, "Find all the images which contain the Queen of England."

The fundamental principle which we are exploiting for our face detector is the Kullback measure of relative information. This measure has the properties that bounds on the classification error probabilities can be proven, and that it leads to feature classes which are better for classification. We apply it toward finding the most informative pixels (MIP), which from a pattern recognition perspective should maximize the class separation. Since the class probabilities are dependent on their neighbors, we model the image as a Markov random field (MRF) and calculate the MIP using a first order assumption.

For the eyes/nose template, the histogram equalized Kullback image (left) and the MIP (right) from a first order MRF are:

[Picture] [Picture]

An example of the output of the face detector is

[Picture]

FAQ

How general is this method?

This method can be used on any classification or recognition problem. Assembling the training databases for the different classes is the most time consuming part.

How does this method compare with other methods?

We tested our algorithm on 4 test databases from 3 different universities (CMU, MIT, Leiden). The results were found to be comparable to other well known methods. Work is currently moving toward providing application specific benchmarks for comparison of the recent methods.

What is the difference between face detection and face recognition?

Face detection has two classes: face or nonface. Face recognition has M classes, where each class represents one person from M individuals. Face detection must discriminate between complex backgrounds and human faces. Face recognition must descriminate between the subtle differences of human faces. Typically, face detection is first performed on an image to obtain the locations of the faces. Second, face recognition is performed in order to determine the identity of each face.

Why use the Kullback relative information?

The Kullback relative information gives a class separation measure which has proven bounds on classification errors. Furthermore, it can be shown that the Kullback relative information is an underlying basis or directly related to other principles such as maximum likelihood, Shannon's mutual information, and minimum description length.

Why use Markov random fields?

In order to calculate prob(FACE | I(x1), I(x2), I(xN)), we would need to model an N dimensional space. By using the first order MRF, we can reduce this to a 3 dimensional space.

Why not use the prob(FACE | I(x1), I(x2), I(xN)) directly for the face detection?

Calculation of the probability of a face functional requires excessive computional time. We reduce the computational time by using a multi-class principal component based method.

Test Methodology?

For our initial experiments we approached the problem from a pattern classification perspective. We have a module which classifies every input template window as either a face or nonface. One important question is "How far can you be from the ground truth face and still have it be classified as a face? Suppose you shift the template by 1 pixel, should it still be classified as a face? What about 2 pixels, etc?" This problem is not localized to face detection but occurs in many if not most classification problems. How much can you distort an object and still have it be classified as the original object? Our initial experimental results are based on allowing a 10% tolerance in the ground truth. This means that the detection rates are based on each individual face having a fuzzy cloud in the sample space - one individual face generates many templates which are classified as faces.

However, the method described above does not give the intuitive solution, which is "How many individuals are located in the image?" When we look at an image, we see individuals, not fuzzy clouds of face probabilities. This question is currently being addressed.

Test Results?

We used four datasets from three universities: Leiden, MIT, and CMU. These are our results: Detection Rate/Number of False Alarms

Leiden 19th Century Portrait Database: 97.4/46

CMU Dataset: 88.3/508

MIT Dataset: 94.1/64

CMU Website Dataset: 84.9/13

Improvements?

We are collaborating with Kah Kay Sung (sungkk@iscs.nus.sg or sung@ai.mit.edu) on integrating neural networks with the Kullback relative information.

Are there any comprehensive direct comparisons available?

We know of no comprehensive performance comparisons between different face detectors. We are currently assembling different application oriented databases for various face detection tasks. Suggestions and contributions are welcome. Please contact mlew@wi.leidenuniv.nl

Face Detection Demo

You can download the latest version of the MIP face detector. Note that you must be using an SGI workstation which has the IRIX 5.3 operating system.

(1) Download the gzipped executable by clicking below

(2) Remove the .bin extension

(3) gzip -d detect_face3.gz

(4) detect_face3

Basically, the executable detect_face3 takes as input any grayscale PGM format (same format as in xv) and after it finds the faces, it writes the output to a PGM image.

detect_face3 [input_image.pgm] [output_image.pgm]

Click here to download the MIP face detector demo (900K)

Real Time Face Recognition Demo

You can download the latest version of the real time face recognition demo. Note that you must be using an SGI workstation which has the IRIX 5.3 operating system.

(1) Download the gzipped executable by clicking below

(2) Remove the .bin extension

(3) gzip -d facecam.gz

(4) facecam

This program allows the IndyCam to perform real time face recognition.

facecam

Click here to download the Real Time Face Recognition demo (40K)