Subscribe
12.13.18 —

Build face recognition directly into your browser

This is a demo project showing how Machine Box tech can be integrated into JavaScript applications.

Facebox can take an image and tell you how many faces it sees, as well as who those faces belong to provided you have shown it a single example previously.

We can use this capability to build additional security into our web apps so we can see how many people are watching the screen and who they are. Using the webcam with some JavaScript and Facebox, we can periodically check to ensure only authorised people can see the information that users consider sensitive.

A working demo of this project is available: Check out theFace Verify project on GitHub

https://medium.com/media/3c60f69cc0e680e117f8ac291854ab9e/href

Use case

Banks don’t want private account details (like the user’s current balance and credit limits etc) being seen by anybody other than the account holder.

The Face Verify technique can be used to automatically hide sensitive information if:

  • Facebox doesn’t recognise the person looking at the screen, or
  • if multiple people are in front of the screen

Ethical implications

This kind of system should only be implemented with user’s consent and if they choose to enable this kind of capability for themselves. In Face Verify, the user has to give the website permission to access the webcam before it will even work, and without a reference image in the first place, Facebox cannot identify them anyway.

Like any technology, we need to consider the ethics of its application carefully so we don’t build tools that are open to abuse, or worst case, terminators that can travel through time to kill people.

Example

In the following screen, I have moved out of view of the Face Verify tool, and the sensitive information is blacked out:

Sensitive information is hidden when I’m not in front of the screen

When I move into focus, the information is revealed:

Once Face Verify is sure that I am looking at the screen — it can reveal the infromation again

The project

The Face Verify project has a JavaScript SDK that provides this capability, and it’s very simple to use:

https://medium.com/media/7b52c451c9be49845036db35ce0ce5e9/href

After spinning up an instance of Facebox (in production you would deploy instances and probably configure them to respond elasticity to demand) you just need to create an instance of the machinebox.FaceVerify class, and call start().

The onSecure and onInsecure callbacks are triggered when the state changes between the page being secure (i.e. only one authorized person is looking at the screen) and being insecure (Facebox doesn’t recognize the person, or if there are multiple people).

The snapshotInterval is how long to wait between checks in milliseconds, so 1000 is one second.

Open source

The solution is open-source (with an Apache license) and you can see the full implementation by checking out the face-verify.js file.

Is this really secure?

The “additional security” headline was chosen deliberately. Obviously you wouldn’t want to rely on this technique as the only security layer but for some circumstances it’s a great way to ensure only the right people are in front of the computer.

There are a number of ways this kind of capability could be circumnavigated, and a few in which it could be abused, which is why we rely on browsers to ensure this kind of thing is only done with the user’s consent.

Build face recognition directly into your browser was originally published in Machine Box on Medium, where people are continuing the conversation by highlighting and responding to this story.