Computer Vision In The Time Of Covid-19

May 6, 2020 by guest author, Sean James

Guest Post: Sanjay Manandhar, ISM Connect

Sanjay Manandhar

Even before Covid-19, limiting touch on public displays was a good hygiene practice.

In digital signage, whenever “interactivity” is mentioned most think of touch interactivity, but few think of social media interactivity or other forms of interaction with the viewing public. While one-to-one touch interactivity, such as downloading a boarding pass or dispensing cash from an ATM, was common practice prior to Covid-19, these interfaces will now need extra cleaning and attention.

Additionally, one-to-many touch interactions on very large digital screens do not typically get a lot of usage, even prior to Covid-19—case in point are large touch screen installations at airports that sit idle after the novelty wears off.

Covid-19 Makes Touch Very Risky

Given that the Covid-19 and other corona viruses remain on various surfaces for extended periods of time, the public now recognizes that touching any surface that is not known to the user is a very risky proposition.

A better architecture of the touchscreen experience might be to encourage viewers to use a much more available and familiar touchscreen device—their own smartphone—and using the smartphone as the input mechanism to collate information on a public (non-touch) display. It could be a direct interaction between the phone and the digital display or an interaction via a social media input, which could create a richer interaction.

Touchless Access Control

Another area that Covid-19 has transformed completely is access control, particularly using fingerprint or palm print and other biometrics to gain access to an office or secure area. No amount of cleaning is going to be fool-proof, nor keep up with such high traffic areas. The better architecture is to allow access without touch but with a similar or higher level of credentialing and security.

In the last 5-6 years, AI has permeated many areas of our everyday lives and is now deemed as important and pervasive as electricity was a century ago. Thus, autonomous drive, robotics, drone technology, and even machine learning around medical data, which has led to better efficacy and disease detection have been successfully demonstrated.

One sub-area of AI that is not well known is one of computer vision, or CV.

Computer Vision for Access Control and Other Use Cases

Computer vision has blossomed recently due to the rise of neural networks, which “learn” just like the human neural circuitry of the brain, hence, the name. CV has also finally reached a point of commercial viability because the computing needed for processing of a multitude of images is economically acceptable for both CPU (traditional computing chips) or GPU (graphical computing units, which are tuned to do a lot of parallel processing of primarily image data) in blazingly fast speeds.

At ISM, the company I co-founded, we have computer vision suitable for solving different use cases, with remarkable success.

Computer Vision for Audience Analytics

Digital out-of-home is notoriously fraught with lack of measurability. Advertisers may get proof-of-performance data of their ads running on screens, but as a media buyer the more important metric is how many people saw the ad or viewed the messaging.

Computer vision can now provide very accurate and actionable audience measurement. With a simple camera, our system can provide metrics for not only how many people watched the content and for how long but we can provide more granular data such as the split of male and female as well as general age ranges.

In particular, when this type of real-time data is plotted over a long timeline (also called longitudinal data), how an audience  behaves and responds becomes very clear, and makes it easy to modify not only the creative that is served, but also the layout of the physical space and offerings for purchase if the space is a retail or merchandising location.

This is facial recognition at its best and the resulting data is aggregated and not personally identifiable.

Computer Vision for Access Control

Another CV use case is with the management of visitors—generally called access control. Most people have experienced having to present their driver’s license or other form of ID to an inattentive security guard in order to gain access to a building—no further validation of the ID or whether the visitor is indeed the presenter of the ID is conducted. This exercise provides a thin veneer of security and access control but does little else.

We believe that a better approach is for hosts to allow their guests to upload their own selfies, which the CV system checks against at the time of entry and automatically unlocks a door or turnstile. This system diminishes the need for interactions with security personnel and subsequently the line that guests must wait in, in order to present their driver’s license. Additionally, this system decreases the need for security guards to touch guest’s ID and for guests to touch keypads or other mechanisms.

The system also provides a precise audit trail of who gained access and if the CV system authorized or stopped a potential entrant. Of course, those that prefer not to use the CV route can wait in line and present their ID, thus, CV provides the “EasyPass” or “TSA Pre-check” type of fast-tracking for those that prefer more convenience and fewer touchpoints, in exchange for digital facial registration.

Computer Vision for Spoof Detection & Thermal Signature

When facial recognition becomes the main mechanism of access control, we need to create a mechanism to protect against attempts to subvert the system. One important countermeasure is the detection of approved human faces that are spoofed by presenting photos of an authorized face which might be printed on paper or displayed on a tablet or smartphone. We use multiple camera techniques, and in some cases, we add a thermal camera, such as an IR (infra-red) camera, to detect spoofing with a very high level of accuracy.

About The Writer

Sanjay Manandhar is Co-Founder & CTO of ISM, which develops and deploys computer vision software products as well as media management products for booking campaigns, managing content and consuming ISM’s computer vision analytics. In addition to licensing its software to customers, ISM applies its products to its own media networks in Minor League Baseball parks, at NASCAR races, and to the solar powered, Sun Media, digital-out-of-home network.