Christian-VillwockAt GDC this year, SensoMotoric Instruments (SMI) showed a couple of new eye tracking demos at Valve’s booth. They added eye tracking to avatars in the social VR experiences of Pluto VR and Rec Room, which provided an amazing boost to the social presence within these experience.

There are so many subtle body language cues that are communicated non-verbally through the someone else’s eye contact, gaze position, or even blinking. Since it’s difficult to see your own eye movement due to saccades, it’s best to experience eye tracking in a social VR context. Without having a recording of your eyes in social VR, you have to rely upon looking at a virtual mirror as you look to the extremes of your periphery, observing your vestibulo–ocular reflex as your eyes lock gaze while you turn your head, or winking at yourself.

eye-tracking

I had a chance to catch up with SMI’s Head of the OEM business Christian Villwock at GDC to talk about the social presence multiplier of eye tracking, the anatomy of the eye, and some of the 2x performance boosts they’re seeing with foveated rendering on NVIDIA GPUs.

LISTEN TO THE VOICES OF VR PODCAST

It’s likely that the next generation of VR headsets will have integrated eye tracking and the goal of both SMI and Tobii to be the primary providers, but neither Tobii or SMI are commenting on any specific licensing agreements that they may have come to with any of the major VR HMD manufacturers. I will say that SMI had some of the more robust social VR eye tracking demos at GDC, but Tobii had more nuanced user interaction examples and more involvement with the OpenXR standardization process in collaboration with the other major VR hardware vendors. You can read more information their integration with Valve’s OpenVR SDK in SMI’s GDC press release.

SEE ALSO
'Rec Room' to Roll Out Full-body Avatars in March

Support Voices of VR

Music: Fatality & Summer Trip

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Kent Bye
  • Rigelleo

    2x boost in performance with foveated rendering??? It’s like from geforce 970 to geforce 1080!! This is the real news! So we have the technology, no need of more development?

    • kontis

      A LOT of more development is needed for decent performance wins:

      14:05

      https://youtu.be/AtyE5qOB4gw?t=845

      • Rigelleo

        This is old, and it was from Oculus. By reading this article it would seem a problem solved. I would like to receive a clarification by the author.

  • I think expecting eye tracking in the next wave of VR releases from VIVE and Oculus might be a little overly-optimistic. I am very strong VR supporter, and at the moment, I am “Tentatively Optimistic” on the future of VR. Killer games and amazing experiences are still rare.

    Unless I’m drastically overestimating the necessary resources involved, I believe Eye-Tracking is a serious addition that will require any current HMD maker to “Double-Down” on their investment in VR. Oculus and VIVE are finally dropping prices down to what the average consumer might pay. The weight and size of each HMD is starting to drop. This will require not only a huge financial investment, but also require a full design of each HMD.

    Unless some enterprising company out there develops a super-cheap, super-lightweight eye tracking chip manufacturers can just pop in their HMD for $10, I doubt any company will want to seriously push back their HMD progress for eye tracking, at least for the next year or two. I’m certain it’ll become a standard, someday down the line, but again, only when Eye Tracking is as simple as adding 2 tiny chips for $10. VR itself is still on shaky footing, this seems like a big step for an industry still doing baby steps.

    That said, I’d totally jump at buying a well supported HMD with eye-tracking.

    • CoffeeBuzz

      Walter Sharrow : “VR. Oculus and VIVE are finally dropping prices”
      I do not see any price drops from HTC/VIVE yet. I wish they would.

      • It’s got to be soon. Valve has already found ways to cut the cost of Lighthouses, as I think they even streamlined the controllers. Unless they plan on adding some KILLER TECH, they have to cut prices to be competitive with Oculus. I guess eye tracking could be that Killer-Tech?

        • Konchu

          I personally also feel we are still paying the new TAX recouping RnD etc. I don’t think these things part wise are quite as high as the are market value. This will normalize with adoption.

    • Rogue Transfer

      Fortunately, it is under $10 for the SMI eye-tracking implementation:

      “Further, we were frankly astonished to learn that the SMI hardware inside is exactly the same thing we saw a year ago at Mobile World Congress. It’s still that little PCB and two skinny wires with tiny cameras at the ends, it can still offer 250Hz, and it still costs under $10 to implement.” – http://www.tomshardware.com/news/smi-eye-tracking-vive-hands-on,33811.html

      Plus, FOVE already launched a dev kit headset for $349 with inbuilt eye-tracking.

      • Well this is good news, albeit still with issue of getting driver support into the hands of developers. Any word about Unreal engine support? Unity? Also, the headsets will have to be re-engineered, as this requires an illumination ring and 2 small camera placements inside. The rig sounds small, and it’s a good sign that they showed this off with the cooperation of Valve. (Would have been a *little* nicer Valve said, “Yes, Eye Tracking, done-deal”)

        Keep in mind that Valve doesn’t make the VIVE, HTC does. So there is the question of HTC’s dedication to upgrades, as opposed to simple price reductions. The VIVE is still well outside of the sweet spot for peripheral sales ($300 or less). Adding new tech isn’t going to speed those price drops along. (even if that tech could massively help framerates and user interaction)

        As for the FOVE, I see they do have a basic system for positional tracking, but nothing for hand tracking. Their 1-camera system is like an older version of the Oculus, and well behind the VIVE and their roomscale tracking. The lack of hand tracking kinda takes them out of the mainstream and more into the realm of “tech curiosity”. It’s something developers should get to acclimate themselves with eye-tracking technology, but it’s not a real player in the market at this time.