Face recognition

From SI410
Jump to: navigation, search

Face Recognition is a technology that allows a computer to identify a person by their facial features. It utilizes biometrics and algorithms to identify and process an individual facial feature.[1]. The data collected is cross-referenced with a database of faces to identify each individual. Companies like Snapchat, Apple, and Amazon have adopted the use of facial recognition in order to augment users' experiences. Face Recognition has been scrutinized for its involvement in personal privacy, incorrect identity recognition, and abuse of power.



In the 1960s, American mathematician and computer scientist Woodrow Wilson "Woody" Bledsoe used a manual device called RAND TABLET to measure facial features like mouth, nose, eyes, hairline as well as others. The system would record the coordinates of these features and input them into a database. When the system was given a new image, the database was able to retrieve the image with the most similar metrics. At the time, the system was limited by the manual work required and the processing power of computers. [2] In the 1970s, A. Jay Goldstein, Leon D. Harmon, and Ann B. Lesk were able to improve on the Bledsoe facial recognition system by adding 21 specific features. The measurements still needed to be taken by hand but their efforts improved the accuracy of the system.[3] In 1988, L.Sirovich and M.Kirby used linear algebra to improve the system and began effort toward what became the Eigenface approach. Later in 1991, Matthew Turk and Alex Pentland, improved this approach through the development of a system that was able to recognize faces in images. This advancement was the beginning of automatic facial recognition. DARPA, otherwise known as The Defense Advanced Research Projects Agency, and the National Institute of Standards and Technology began the FERET project, Face Recognition Technology, to encourage the use of commercial facial recognition and innovation in the facial recognition field. A database of 2,413 still facial images representing 856 people was created as a part of the project, which included high-resolution 24-bit color images. [4] In 2002 during Super Bowl XXXV, law enforcement authorities used facial recognition technology to analyze the faces of patrons in attendance at the football game. They intended to mitigate risk and maintain the safety at the game By identifying criminals as they entered the stadium. Although they did identify petty criminals, the utilization of this technology at the event was seen as a failure due to the number of false positives and negative criticism. The Pinellas County Sherriff’s Office allowed officers to utilize a photo forensic database which included images from the state’s Department of Highway Safety and Motor Vehicles in 2009. Officers were able to take photos of suspects and cross-reference them with archives from the database using facial recognition technology. By 2010, The social media platforms Facebook implemented a facial recognition technology that allowed users to automatically tag other users in their photos. This became controversial due to the fact that many users, as well as the media, expressed concerns about personal privacy. In 2011, Osama bin Laden was identified as dead after a US Navy seal raid, Using military-grade facial recognition technology.


In recent years, facial recognition technology has improved. It is being used equally in different fields, most notably security. Recent developments have even made systems that can recognize differences in identical twins. The 2002 Face Recognition Vendor Test shows that the technology of face recognition capabilities is increasing rapidly. The rate of error in facial recognition has gone down 50%[5] according to this test.

FRVT 2002

The Face Recognition Vendor Test of 2002 is a test that inspected the accuracy of facial recognition and explored areas for future development. The FRVT 2002 tested each face recognition system on its ability to identify correctly, verify the right individual, and "watch-list screening." Watch-list-screening is the process of facial recognition software identifying a particular individual and who that individual is. The test resulted in an 85% identification rate, 96% verification rate, and a 77% detection rate for the top few facial recognition systems.[6]

3D recognition systems

Face recognition technology is gaining popularity, and new features and capabilities are surfacing to improve the accuracy of recognition systems. One particular feature was developed by the Bronstein brothers. Michael and Alexander Bronstein are electrical engineering graduate students and identical twins. As identical twins, they were always being mistaken for one another. Facial recognition software could not tell them apart, so their information was not safe. They created new facial recognition features that utilizes 3D technology to solve the identical twin problem.[7] This feature enhances the software because it maps specific features to the face instead of processing the face as a single image.[7] The technology is resistant to the effects of lighting and makeup in recognizing faces.

Major Uses

Facial recognition has proven to be a very useful technology. Being able to access an individual's face has changed the way businesses and institutions handle everyday security concerns.


The most prominent form of face recognition that people see is on social media. A research group at Facebook developed DeepFace, a facial recognition algorithm using machine learning, to identify faces with 97% accuracy.[8] Facebook has implemented this software to automatically tag a user's friend in their photos using face recognition software. [9]


Snapchat uses facial recognition in order to allow users to add filters to their Snapchat photos. The app recognizes a face, and then users can add a dog filter, flower crown, or glasses to their photo.


Some dating websites also use facial recognition technology to match people based on facial feature preferences. [9] For example, the app "Dating.ai" allows the user to upload a photo of someone that they would like to date, and using facial recognition technology, it would match the user with another user that has similar facial features. [10]


Law enforcement uses face detection to track down criminals, improving the crime rate.[9] The US Department of State manages a facial recognition database with over 117 million Americans that the FBI uses to aid investigative efforts. The database contains mostly photos from state-issued licenses and IDs. [11]

iPhone X

Today, facial recognition is integrated into the new iPhone X. The iPhone X uses Face Id, a technology that detects a user's face when looking at the phone and unlocking the device if the face is identified as the users. While scanning one's face, the iPhone uses infrared and visible light scans to identify one's face. These biometric authentication security measures are uniquely designed for the iPhone owner. This feature on the iPhone X relies on many unique facial features such as lighting conditions and accessories to enhance the user experience. The user must look at the phone attentively in order for the facial recognition technology to work. Apple claims that their Face Id technology is secure and that there is a 1 in a 1,000,000 chances that another face will match as another user. [12]

Types of Facial Recognition

There are many different ways that Facial Recognition has made itself a prominent and rising technology in our society.

Algorithmic Facial Recognition

Facial recognition by algorithm leverages general metrics of facial proportion in conjunction with the identification of facial landmarks to determine when a face is visible in a camera frame. Many algorithms analyze the relative positions of the most notable features of the face (eyes, mouth, nose, jaw) as well as their shapes and locations relative to each other. size, and/or shape of the eyes, nose, cheekbones, and jaw. Algorithmic facial recognition can also be used to identify the same person in a suite of photographs. [13] Once the relevant information has been gathered, the algorithm can scan the photographs for the same relative proportions, features, etc. with fairly high accuracy. This type of facial recognition is meant only for static photographs, and would not be effective in identifying people in videos or authenticating a person's identity in real time.

Thermal Facial Recognition

Facial recognition using thermal technology is a less common form of facial recognition. In this case, a thermal camera captures a thermal image -- and the temperature distribution of the scene as a result. In such images, faces of different people present differently and the temperature of faces is more pronounced than the backgrounds of thermal images. This allows for face detection regardless of lighting conditions -- which means it can be done in the dark! A person's facial temperature distribution is also more or less unique, which makes it ideal for identifying individuals. Using thermal images also helps in discriminating between real people and photographs, which can be very useful for identity authentication. Additionally, the use of thermal images for facial recognition addresses growing privacy concerns. Facial temperature distributions are very hard for the human eye to parse, meaning that anyone looking for a certain person in a database of identities will have a difficult time, but the thermal matching system will have no trouble.[14]

Infrared Facial Recognition

The most common form of infrared facial recognition is the iPhone X Face ID. When you raise your phone to your face, the software calculates the amount of light needed to light up your face. Rather than shine a flashlight at you, it shines infrared light onto your face -- a wavelength imperceptible to the human eye. The points of infrared light help the software create a map of your face, based on the amount of reflection and diffusion from your skin. This allows a very precise image of your face to be created -- including unique features such as jaw shape, distance between eyes, width of face, etc. This can be matched to the map of your face you made when you set up Face ID to verify your identity. The similarity between the two maps is calculated on a scale from zero to one. If the score is close enough to one, the faces are deemed similar enough and your identity is validated. The huge benefits of infrared facial recognition are that it allows identification in the dark (the light it uses is infrared!), often works with sunglasses (the infrared light penetrates the lenses!), and can be used on live and moving people (the infrared light creates a map rather than a static picture!). [15]


  • Increased Security - Facial recognition is mostly used for security purposes. Most business have adapted this technology to add more security to their confidential data and files. [16]
  • Business - Facial recognition is also be used in marketing. The software can determine attributes such as gender, age, and ethnicity. Retailers use this information for targeted advertising.[16]
  • Airport Security - In the near future, airports may be adding facial recognition to their security systems. This would streamline the security process at airports, making travel more efficient and safe.[16]
  • Event Feedback Research - Facial recognition technology allows for better understanding human's facial expressions which in turn can be used to figure out what a particular person is thinking at that particular moment. [17]

Ethical Concerns

As facial recognition technology has improved, more users have developed opinions on the nature of the technology.


While facial recognition technology aids security measures, many individuals fear that it is an invasion of privacy. People fear false positives, meaning the facial recognition system could wrongly identify an individual [18]. This would have many repercussions including: allowing them access to someone else's private information and/or falsely accuse someone of a crime they did not commit.

Multiple Identities

There are a plethora of issues related to what a person should look like, but many individuals have plural selves or multiple identities.[19] Yet, facial recognition software only allows for one identity to be recognized and allowed access to the information it is protecting. For people who have multiple identities, they will only have access to certain information, regarding themselves, when they choose to identify a certain way. This raises many ethical dilemmas about who has the right to this information and why only certain individual's identities should be allowed to access their protected information.


Biases of programmers who create the IA for these technologies can be included in facial recognition software. While it is the hope that this is not the case, more often than not, it is. A common issue seen here is racial bias, which can slip, unintentionally, into algorithms. "The engineer that develops an algorithm may program it to focus on facial features that are more easily distinguishable in some races than in others – the shape of a person's eyes, the width of the nose, the size of the mouth or chin." [20] This is where we begin to see racial biases, and though they are seemingly harmless to a developer, once launched to the public, the ethical implications become obvious. Another bias that is seen is self-bias. This being when algorithm creators think about what they look like rather than what users look like. This can lead to certain users facing issues in using facial recognition if they do not look like the designer.


The abuse of power with this new technology is a large ethical concern. For example, a parent or spouse could track their child or significant other without their knowledge. This again addresses the issue of privacy invasion.[21] A parent may also program their facial recognition into their child's phone without them knowing, allowing the parent to have unauthorized complete access to all information on the device.


  1. “Biometrics Expert Delivers Lecture on Facial Recognition.” RAND Corporation, www.rand.org/natsec_area/products/facialrecog.html.
  2. “History of Face Recognition - Facial Recognition Software.” FaceFirst Face Recognition Software, 14 Dec. 2017, www.facefirst.com/blog/brief-history-of-face-recognition-software/.
  3. “History of Face Recognition - Facial Recognition Software.” FaceFirst Face Recognition Software, 14 Dec. 2017, www.facefirst.com/blog/brief-history-of-face-recognition-software/.
  4. “History of Face Recognition - Facial Recognition Software.” FaceFirst Face Recognition Software, 14 Dec. 2017, www.facefirst.com/blog/brief-history-of-face-recognition-software/.
  5. D. Voth, "Face recognition technology,"IEEE Intelligent Systems, vol. 18, no. 3, pp. 4-7, May-Jun 2003.
  6. Flanagan, Patricia A. “Face Recognition Vendor Test (FRVT) 2002.” NIST, 25 Aug. 2016, www.nist.gov/itl/iad/image-group/face-recognition-vendor-test-frvt-2002.
  7. 7.0 7.1 Bronstein, Alexander, et al. “Three Dimensional Face Recognition.” Numerical Geometry of Non-Rigid Shapes Monographs in Computer Science, 2008, pp. 261–275., doi:10.1007/978-0-387-73301-2_13.
  8. Russell Brandom (July 7, 2014), "Why Facebook is beating the FBI at facial recognition", The Verge
  9. 9.0 9.1 9.2 “Top 8 Ways Facial Recognition Software Is Being Used Today.” Tech Guru, LLC, 14 July 2015, www.techguruit.com/top-8-ways-facial-recognition-software-used-today/
  10. https://nakedsecurity.sophos.com/2017/06/23/dating-app-boss-sees-no-problem-on-face-matching-without-consent/
  11. "Here's How Many Adult Faces Are Scanned From Facial Recognition Databases" Fortune, 18 October 2016. http://fortune.com/2016/10/18/facial-recognition-database/
  12. Fleishman, Glenn. “Face ID on the IPhone X: Everything You Need to Know about Apple's Facial Recognition.” Macworld, Macworld, 25 Dec. 2017, www.macworld.com/article/3225406/iphone-ipad/face-id-iphone-x-faq.html.
  13. https://en.wikipedia.org/wiki/Facial_recognition_system#Traditional
  14. Ma, Chao, et al. “Adapting Local Features for Face Detection in Thermal Image.” Sensors (Basel, Switzerland), MDPI, 27 Nov. 2017.
  15. Ghiass, Reza. “Infrared Face Recognition: A Comprehensive Review of Methodologies and Databases.” Pattern Recognition, Pergamon, 28 Mar. 2014.
  16. 16.0 16.1 16.2 “Face Recognition Is Transforming the Security Industry.” Kairos, 31 Aug. 2017, www.kairos.com/blog/face-recognition-and-the-future-of-security.
  17. https://blog.capterra.com/3-major-benefits-of-facial-recognition-technology-for-event-planners/
  18. “False-Positive.” Dictionary.com, Dictionary.com, www.dictionary.com/browse/false-positive.
  19. Cocking, Dean. (2008). “Plural Selves and Relational Identity: Intimacy and Privacy Online,” in Information Technology and Moral Philosophy, edited by Jeroen Van Den Hoven and John Weckert. Cambridge University Press, pp. 123-41.
  20. "Researchers flag up facial recognition racial bias". 2016. Biometric Technology Today.
  21. K.W. Bower. "Face recognition technology: security versus privacy" IEEE Technology and Society Magazine, vol. 23, no. 1, pp. 9-19, Spring 2004