VFace: on-device face biometrics

Problem

Biometric login lives or dies on two numbers: how often it lets the wrong person in, and how often it locks the right person out. It also has to run on whatever phone the customer owns, in whatever light they happen to be in.

What I did

I led VFace, Veridium’s face biometrics stack. I trained the models for facial landmarks, gaze, head pose and face recognition on MobileNet backbones, using quantisation-aware training so they kept their accuracy once converted to 8-bit TFLite for phones, tablets and the browser. I used uncertainty estimates to make landmark detection robust. I also worked on pose-based liveness: tracking 3D head pose on the device during capture, alongside gaze, to repel presentation attacks such as video replays, print-outs and masks. Every model went through automated ML checks in CI with CML before it shipped.

In the clip above, everything except the overlays has been mosaicked so the person stays anonymous. Green box with confidence: face detection. Dots with circles: landmarks, each circle showing the model’s own uncertainty. Red, green and blue axes: 3D head pose. Magenta and cyan arrows: per-eye gaze.

Result

False rejection fell from 3% to 0.4% at a false acceptance rate of 10⁻⁶, and normalised mean error in landmark detection halved.

Face recognition was also tested independently by NIST, the benchmark buyers and governments check. Our 2023 and 2024 submissions held accuracy level while getting much leaner: against the late-2022 entry, template generation became 39% faster (44 to 27 ms), comparison 45% faster and the library 36% smaller (39 to 25 MB), with fewer false non-matches on mugshots, mugshots taken 12+ years apart and hard profile visa-to-border pairs. The full results are public on NIST’s site.