The COVID-19 pandemic has led to many changes in society around the world and one of the biggest changes is that many people are now required to wear masks, both indoors and out.
Since these facial masks cover up half of our face, they pose a problem for existing facial recognition systems.
As we have reported previously, the use of facial recognition technology is increasing around the world, also as a direct result of the COVID-19 pandemic. More and more businesses are turning to contactless solutions that include biometric authentication such as facial recognition or iris recognition to reduce the number of touchpoints for customers and staff.
From international travel to access and authentication, businesses are looking at ways to keep their staff and customers safe. Reducing the number of touchpoints an individual must make on a day-to-day basis is one way that businesses are helping to tackle the spread of COVID-19 and potentially other viruses that can be passed by close contact.
The problems of face masks for facial recognition technology
According to a study carried out by the National Institute of Standards and Technology (NIST), facial recognition systems that were developed pre-pandemic are easily flummoxed by face masks.
The NIST team explored how well each of the algorithms was able to perform “one-to-one” matching, where a photo is compared with a different photo of the same person.
The research team digitally applied mask shapes to the original photos and tested the algorithms’ performance. Because real-world masks differ, the team came up with nine mask variants, which included differences in shape, colour, and nose coverage.
“We can draw a few broad conclusions from the results, but there are caveats,” said Mai Ngan, a NIST computer scientist and author of the report. “None of these algorithms were designed to handle face masks, and the masks we used are digital creations, not the real thing.”
Some key findings of the report include:
- Algorithm accuracy with masked faces declined substantially across the board. Using unmasked images, the most accurate algorithms fail to authenticate a person about 0.3% of the time. Masked images raised even these top algorithms’ failure rate to about 5%, while many otherwise competent algorithms failed between 20% to 50% of the time.
- Masked images more frequently caused algorithms to be unable to process a face, technically termed “failure to enroll or template” (FTE). Face recognition algorithms typically work by measuring a face’s features — their size and distance from one another, for example — and then comparing these measurements to those from another photo. An FTE means the algorithm could not extract a face’s features well enough to make an effective comparison in the first place.
- The more of the nose a mask covers, the lower the algorithm’s accuracy. The study explored three levels of nose coverage — low, medium and high — finding that accuracy degrades with greater nose coverage.
- While false negatives increased, false positives remained stable or modestly declined. Errors in face recognition can take the form of either a “false negative,” where the algorithm fails to match two photos of the same person, or a “false positive,” where it incorrectly indicates a match between photos of two different people. The modest decline in false positive rates show that occlusion with masks does not undermine this aspect of security.
- The shape and color of a mask matters. Algorithm error rates were generally lower with round masks. Black masks also degraded algorithm performance in comparison to surgical blue ones, though because of time and resource constraints the team was not able to test the effect of color completely.
How NEC is tackling the issue of facial recognition and mask wearing
In January 2021, Reuters reported on the work being carried out by NEC to tackle the issues faced by facial recognition solutions by people wearing face masks.
NEC is ahead of the curve because we were already working on a recognition system capable of dealing with masks before the pandemic started. Mask wearing is common in Japan, with individuals choosing to use them either when ill or suffering from allergies.
“Needs grew even more due to the coronavirus situation as the state of emergency (last year) was continuing for a long time, and so we’ve now introduced this technology to the market,” Shinya Takashima, assistant manager of NEC’s digital platform division, told Reuters.
“Touchless verification has become extremely important due to the impact of the coronavirus,” he said. “Going forward we hope to contribute to safety and peace of mind by strengthening (efforts) in that area.“
The system determines when a person is wearing a mask and hones in on the parts that are not covered up, such as the eyes and surrounding areas, to verify the subject’s identity. Users register a photo of their face in advance.
Verification takes less than one second and with an accuracy rate of more than 99.9%.
Who can benefit from facial recognition technology that can recognise mask wearers?
This technology was launched by NEC in October 2020 with particular interest from the airline sector. Both Lufthansa and Swiss International Airlines are customers and already benefiting from the technology that allows customers to keep their face masks on while transiting through check-in procedures with those airlines.
The system, however, has much wider applications and could be used at security gates and doors in office buildings and other facilities. NEC is trialling the technology for automated payments at an unmanned convenience store at our Tokyo headquarters.
NEC also provides the facial-recognition technology used by Customs and Border Patrol at U.S. airports.
According to Benji Hutchinson, a vice president with NEC’s U.S. division, the company’s algorithms have always tested on face masks because they are commonly worn in Asia during flu seasons. “Masks are nothing new to us, but that doesn’t mean it’s all perfect,” Hutchinson told Wired. He said NEC is advising customers, such as CBP, to make their own decisions about the technology for now.
Another NEC biometric authentication device scans not only the faces of users but also their irises, according to a report in Japan Times. The rate of error is said to be under one in 10 billion, and the scan only takes around two seconds, according to the Tokyo-based electronics company.
Touchless experiences are driving technological advances
The COVID-19 pandemic is proving to be a big driver of technological change. Many businesses are looking at opportunities to reduce the number of contact points for customers and staff and biometric technology is at the heart of this.
In March 2021, Disney World launched a trial of facial recognition technology to create a touchless experience but also to reduce the time people spend queuing in their theme parks.
FIRST LOOK: Going Through New Facial Recognition Park Entry Procedure at Walt Disney Worldhttps://t.co/nLTFTHkntv pic.twitter.com/JBlrEcqzeg
— WDW News Today (@WDWNT) March 23, 2021
Both solutions help to reduce the potential spread of viruses like COVID-19. Fewer touchpoints and less time spent in queues with large crowds both present an opportunity for businesses to improve the overall safety for customers and staff while at the same time improving the customer experience.
NEC and facial recognition
In August, NEC’s facial recognition algorithm again placed first for accuracy in two key categories of the National Institute of Standards and Technology’s (NIST) most recent Face Recognition Vendor Test (FRVT) 1:N Identification.
- #1 in Mugshot Identification
- #1 in Border Security Identification
This builds on two decades of success in which NEC has consistently outperformed the competition in terms of both accuracy and speed, even when faced with demographic differentials like race and sex, or when subjects are wearing facemasks for coronavirus.
Over the last decade, our algorithms have consistently placed at the top of NIST’s ranking on several factors primarily due to our massive investments in research and a sharp focus on the needs of our customers. Ensuring algorithms are accurate is essential to helping our customers fulfil their important missions and building public trust in the use of these technologies.
Here in New Zealand, facial recognition technology that can identify people wearing face masks could prove a great addition to the technology already used by Immigration New Zealand in the New Zealand Electronic Travel Authority. Read more about the NZeTA in our recent case study.
Summary
Once international travel begins to open up once more, face masks will likely become a mandatory part of the airport and travel experience. Reducing the number of contact points for customers passing through airports is going to be a crucial element in both helping to keep customers safe and reducing the time people spend queuing throughout the airport journey including check-in and border control.
The technology developed by NEC will help to create a smooth experience at airports around the world, while allowing travellers to leave their masks on throughout the process, helping to tackle the spread of COVID-19 and potentially other viruses that can be passed by close contact.
Related articles
Let’s Connect
Complete our quick and easy contact form and one of the team will be in touch.