Those new glasses make you look completely different – especially to face recognition software.
A team of researchers from Carnegie Mellon University has fooled face recognition algorithms using the oldest trick in the book: a pair of fake glasses.
By printing bespoke patterns onto the front of the frames, they enabled wearers not only to obscure their identity but to impersonate people who look completely different, at least in the eyes of the algorithms.
A white male researcher wearing the glasses was able to pass for American actress Milla Jovovich while a South-Asian female colleague was digitally disguised as a Middle-Eastern male. Both tricked commercially available face recognition software Face++ with a success rate of around 90 per cent. The system wasn’t perfect, however: a Middle-Eastern male trying to use the glasses to pass as white British actor Clive Owen only succeeded 16 per cent of the time.
Previous tricks to confuse face recognition software have included camouflaging makeup and a light-emitting “privacy visor”, but these are easy for a human to spot.
“With some refinement, our glasses would just look like someone had frames with a normal tortoiseshell pattern,” says Mahmood Sharif, co-creator of the glasses. This means the jazzy frames could fool computers without alerting humans to the subterfuge.
The patterned glasses work by exploiting the neural networks used in face recognition systems. Neural networks don’t rely on the same features that humans do to recognise faces. The systems often focus on things like the colour of different pixels and slowly piece together a best guess of who’s in the shot by comparing it to other, similar images. If just a small area of the face has been changed, it can completely mess with the attempted recognition – which is why the computer system can confuse two people who in fact look very different.
“We’re starting to find that neural networks don’t always have the flexibility that we once thought they had,” says Sharif. “So only a few small targeted changes can have a large overall effect in tricking the software.”
By analysing the differences between a specific face and the one they wanted it to be mistaken for, the team could work out how to confuse the software.
If the face fits…
They designed bespoke glasses with a pattern that changes how the system interprets the wearer’s face. The frames essentially overlay the face with pixels that perturb the software’s calculations in just the right way that it misidentifies the person as another specified face in its database – Milla Jovovich, for example. To a human, the frames just look like a colourful tortoiseshell design.
Face recognition systems are slowly appearing everywhere. Fifty per cent of people living in the US are in a police face recognition database, while Facebook uses face recognition software to automatically tag people in pictures posted on its site. Some shops are even using the technology to identify high-value customers.
Fooling face recognition software could help protect an individual’s privacy, but it could also conceivably be used to commit identity fraud.
The new glasses, however, would be “limited to applications with less security,” says Raghavendra Ramachandra from the Norwegian Biometrics Laboratory. They might stop a shop from tracking your movements, but they’re less likely to fool border control – who always ask you to remove your spectacles.
Journal reference: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, DOI: 10.1145/2976749.2978392