Face recognition with eigenfaces

Problem

Recognising a face from a photo first needs the faces to line up: the same size, orientation and position. Only then can a method like principal component analysis compare them. The task, for the Applied Mathematics course in the Master in Computer Vision at the Université de Bourgogne, was to build the whole chain on photos of our own class.

What I did

With Pamir Ghimire, under Prof. Désiré Sidibé, I built the pipeline in MATLAB. Each 320 × 320 photo comes with five feature points. We map them onto predefined positions in a 64 × 64 image with an affine transform, an over-determined system of 10 equations and 6 unknowns solved by least squares with SVD. The target positions are then updated to the average of the normalised points, and the loop repeats until they move less than a pixel, which takes about six or seven iterations.

For recognition, every normalised face becomes a 4,096-long vector. Since there are far fewer images than pixels, we computed the eigenvectors of the small p × p matrix instead of the full 4,096 × 4,096 covariance, recovering the eigenfaces from it. Each face is projected into this PCA space, and a test face is matched to the training face with the smallest distance. Of five photos per person, three went to training and two to testing. I also built a GUI to pick the training and test sets, choose a query face and browse the ranked matches.

Result

With 97 training images and 57 test images, and components kept up to 95% of the variance, the best match was the right person 53% of the time. Allowing the correct person anywhere in the closest matches, accuracy rose to 66% within the top 5 and 81% within the top 15. Most failures came from faces in a similar pose to the query: with so few training images per person, plain PCA tends to match orientation rather than identity.