MindMaze, a neurotechnology startup, is developing a simple, low-cost approach to face-tracking called MASK—claimed to be compatible with any VR headset—that can map significantly more life-like expressions onto your virtual avatar.

I recently met with MindMaze at their San Francisco working space to test Mask for myself and learn more about it.

Since optical, computer-vision based face-tracking is made more challenging in VR due to the headset blocking a significant portion of the face, Mask instead employs low-cost electrodes around the periphery of the headset’s foam padding to sense the electrical signals generated when you move muscles in your face.

Photo courtesy MindMaze

Speaking to MindMaze CEO Tej Tadi, I was told that the embedded sensing hardware is very low-cost and adds little to the headset’s overall cost of manufacturing. The challenge is not the hardware, Tadi says, but the software which is tasked with interpreting the input.

Each graph here represents the electrical activity detected by the electrodes in the headset. Decoding this data into something useful is the challenge. | Photo by Road to VR

The electrodes, which make direct contact with your face through the headset’s foam interface, only measure electrical amplitude. With eight electrodes embedded on the prototype version of the Mask that I tried, the only thing the computer “sees” is eight incoming data streams that go up or down on a graph. Tadi says that MindMaze has applied their neurotech knowledge to create an algorithm that can read that data, and, from it, extract a set of facial expressions.

Neuro-bamboozling vs. the Real Deal

Now I should be clear up front that I’m extremely skeptical when it comes to neuro-bamboozling: that’s when companies use ‘neuroscience’, ‘neurotechnology’, and other brain buzzwords to make it seem like they’re doing something more significant than they really are. One oft-seen example of neuro-bamboozling is when a company might tell you something like, “you can fly a drone with your mind!,” which, 99% of the time, means they’re going to slap some electrodes on your head and tell you to “concentrate”, which will make the drone go up, and then to “relax” which will make the drone go down. It’s highly binary, and ultimately not very useful.

Nearest I can tell for MindMaze—at least for Mask, as I haven’t seen their other products—they are the real deal. When I strapped on the headset to try Mask for myself, even without any calibration, a range of canned expressions that I made were quickly reflected on the face of an avatar that represented me in the virtual world. When I smiled, it smiled. When I frowned, it frowned. When I winked, it winked. It was easily the best calibration-free tech that I’ve seen of this sort.

I was told the prototype presently supports 10 different facial expressions. Because they are canned poses, which means Mask can only provide approximations of your expressions; it won’t be able to capture the unique movements of the face that make you, you, but it does cover the basics.

SEE ALSO
Vive Eye-tracking & Corrective Lens Add-on Confirmed Compatible with Vive Pro

In addition to the 10 current expressions, the team says they’re working to suss even more expressions out of the data, including the potential for rudimentary eye-tracking, which wouldn’t be precise enough for things like foveated rendering but could be good enough for expression mapping.

Continued on Page 2: Not Perfect, But Promising »

1
2
Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • psuedonymous

    I;m afraid this is indeed a case of ‘neuro-bamboozling’: what is going on here is absolutely NOT Electroencephalography, but ElectroMYOgraphy. They are measuring muscle contractions, not neuron activation. It;s double infuriating because their tech is functional and useful (and measuring muscle contractions is the correct way to do this), but they go and muddy the water by using Buzzword Bullshit to market it.

    • benz145

      They didn’t claim that this is “brain reading” technology, they were clear that the tech is measuring the electrical signals in your face and not reading your mind.

  • VRMaven

    Despite being a billion dollar startup, I wouldn’t hold your breath on them having anything on the market this year- especially with that dorky ear clip

  • wheeler

    Something like this mixed with eyetracking will take social VR experiences to the next level.

  • Strawb77

    “..look as if you’re sneering at them (due to misinterpretation of your facial expressions) when you’re actually smiling.”
    one application i can see with this would be with [ie:] autistic people, who have great difficulty reading [and presenting] facial expressions- if the mask could be made haptic, for instance, you could show what expressions `feel` like as you make them.
    possibly could help serious burns patients or others who have had skin grafts also.