Understanding Facial Recognition Software

A picture of a woman's face overlaid with a facial recognition pattern.
iStock Photography

Soon, the contents of your wallet - IDs, credit cards, even your computer passwords - could be replaced by your face.​

You already use facial recognition software in your life. It’s how Facebook knows to suggest you tag your friend Brian in a photo. It’s how iPhoto groups all the pictures of your Aunt Linda together. And it’s how law enforcement looks for suspects in a crowd.

But how does it work? Once it recognizes your face as, well, a face, facial recognition software identifies certain points on it (the spot between your pupils, for example) and measures those in precise increments, down to the submillimeter. Then the software uses those measurements to create a template, or pattern, of your face. That template is then compared with others already stored in the system—whether it’s the dozens in your iPhone or the estimated 52 million in the FBI’s database—to look for matches.

Of course, it’s not always that easy. Many things can trip up the process, such as bad image quality, objects that obscure the face (sunglasses and hats, for example), insufficient lighting, or a face that’s mostly in profile. But there’s no question that facial recognition software is already a part of our daily lives—and growing.

Besides its most visible uses in Facebook and programs like iPhoto, facial recognition software is moving into other areas. Computer users can log onto their computers with their face—not a password—with Windows Hello, a feature of Windows 10. Apps like FaceCrypt let you access your iPhone and its data with your image, not a passcode. China has rolled out ATMs that dispense cash with just your smile.

(By the way, those Snapchat lenses that add a crown of flowers or a puppy nose to your selfie? The company says those use object recognition software, not facial recognition. The difference, it stresses, is that the software recognizes that there’s a face there, but doesn’t try and identify whose face.)

Facial recognition software is also increasingly being used in the business world. Retailers use the technology to spot return customers in shops and give them special offers. Some companies use it to keep track of employees’ attendance—instead of punching a time card, employees look into a camera. In the U.K., the British Broadcasting Corporation has used facial detection software (a close cousin to facial recognition software) via webcams to see how online viewers react to advertisements—did they smile, frown, look annoyed?

Facial recognition software is also used to create the incredibly lifelike animations in today’s movies. Studios such as Pixar and Disney use the software to capture the intricacies of human expressions, such as the way your nose crinkles when you smell something bad. Animators then use those measurements to create characters with remarkably lifelike characteristics and movements.

The most well known use of facial recognition software, of course, is in law enforcement. Police and federal agencies use it to scan crowds for wanted men and women, check suspects against mug shots, and protect borders. In North Carolina, it’s been used to check the DMV database for missing persons. And police in a Virginia beach town installed cameras during the busy summer and used facial recognition software to look for people with outstanding arrest warrants. At airports and border crossings, facial recognition technology is used to check for fake passports—does the traveler standing there match the passport photo on file?

For all the fans of facial recognition software, there are critics too. There are concerns about privacy and the potential misuse of stored information, but experts say the benefits outweigh the drawbacks. “People will exchange some degree of privacy for enhanced security, particularly if it’s protecting their financial assets. One good example would be the use of facial recognition at retail point-of-sale terminals. Instead of signing your name, you’d simply look into the camera and press a button. Your image would become the authorization for payment,” says Pete Smith, senior manager, Product Knowledge and Training at TE Connectivity, a global technology leader of connectivity and sensor solutions. “Online purchases could be handled in exactly the same way using the camera on your phone or computer. It’ll be extremely difficult for crooks to beat this approach.”

Written by: Nancy Gupton