Eonite Claims to Have Solved Inside-out Positional Tracking for VR and AR

51

Lots of companies have made this claim actually, but Eonite specifically says they have the “world’s most accurate, lowest latency, lowest power consuming software to democratize inside-out positional tracking for VR and AR.”

Inside-out positional tracking—the ability to precisely determine where an object is in space using only sensors mounted on the device itself—has been an obvious need but an elusive challenge for the VR and AR industries. AR in particular requires such accurate and low latency tracking that virtual objects can appear fixed to the real world.

We’ve seen very impressive inside-out positional tracking before on Microsoft’s HoloLens—a $3,000 full-blown Windows PC on your head—which houses a bevy of sensors. But today Eonite is announcing their inside-out tracking solution, and claims that it supports low-latency, high accuracy headtracking using just commodity depth sensors and tablet-class processing power

eonite-vantage-head-tracker-(7)That’s potentially revolutionary for the VR and AR industry if true. And while we’ve haven’t gotten our hands on the tech just yet, Eonite has attracted the attention of Silicon Valley venture capitalists and angel investors who dropped $5.25 million on the company in a Seed investment in 2016. Among the investors is Presence Capital and The VR Fund, who specialize in VR & AR tech investing.

The company’s tech is not hardware, but general purpose software for achieving high performance inside out tracking. “It’s not a future promise. It works,” Eonite CEO Youssri Helmy said, speaking with Road to VR. He says Eonite’s tracking is capable of sub-millimeter accuracy and just 15 milliseconds of motion-to-photon latency; both under the threshold of what’s considered high enough performance for VR tracking.

SEE ALSO
Virtuix Omni One VR Treadmill Nabs Support for Some Big VR Games

eonite vantage head tracker (1)The company is calling the capabilities of the tracking ‘homescale’, to suggest that it can enable tracking across a multi-room, home-sized space, and is tuned to track well given the sort of objects you might find in a common home (furniture, shelves, doors, thin etc). Helmy says that the tracking tech integrates IMU and RGB data, and can work with “any depth sensing, from high def stereo, time-of-flight, rolling shutter, global shutter. Anything. The software doesn’t have much to do with the camera.” The software is also said to support both static and dynamic real-time obstacle detection for avoiding things like walls and pets.

Helmy says the tracking software is built on years of work on artificial perception in robotic and consumer applications by co-founders Dr. Anna Petrovskaya and Peter Varvak. “It’s the same core technology for tracking robots as tracking headsets,” he said. “The tech they had blew me away [when I first saw it].”

But it isn’t just for tracking. Eonite is working on a Unity SDK which the company says will allow developers to bring real-time 3D scanned data from the user’s environment into the virtual world for mixed reality and AR applications, including support for persistent virtual content, shadows, and occlusion.

While the company is primarily pitching the tech for AR and VR tracking for now, it’s also said to be a solution for other industries like automotive, robotics, and manufacturing. The first product using the company’s tracking will launch in the first quarter of this year. Helmy says.

While Eonite’s technology sounds promising, 2016 saw demonstrations of major progress on inside-out tracking by a number of companies. First was the aforementioned HoloLens, followed by the impressive tracking of Qualcomm’s VR reference headset, along with Oculus in October who showed off highly functional inside out tracking on the Rift ‘Santa Cruz’ prototype. Inside out positional tracking is likely to be a dominant theme of AR and VR in 2017, and if truly solved by any of these players, will mark a major next step for the industry.

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • Sponge Bob

    Ben,

    on this blog you write about HTC Vive Lighhtouse tracking as some kind of future standard and, at the same time, you write about “inside-out tracking” which, IF works, will instantly eliminate HTC and Oculus tracking devices altogether, and tons of other companies too..

    So which is it ? HTC Lighthouse (or Oculus camera based) OR this magical sub-mm inside-out tracking gimmick you never tested yourself in a location of YOUR choosing ?

    From basic physics perspective, if you place yourself in a big hall with non-descriptive (e.g. white painted) walls this thing just can’t work – the depth sensors are active time-of-flight IR devices – only work to some distance and give you only about 1 cm precision, and the cameras will have no features to detect and analyze
    When talking about “inside-out” tracking it helps to remember that it DOES depend A LOT on the outside environment – e.g. if placed in open space it cant work in principle

    My guess if you have enough of those depth sensors looking in each direction, plus a few cameras, then it can work in a small room – thus “room scale” :):):):):)

    • user

      every tracked object (controllers, etc) needs depth sensing cameras. i wonder how long it takes to bring the cost of the cameras down so that inside out tracking is cheap enough to replace the lighthouses.

      • Sponge Bob

        Of, dude, your fantasy really has no limits…
        You should be writing science fiction novels, not commenting on this blog

        And, btw, why do you think controllers need depths cameras or any cameras for that matter ???
        Even assuming your science fiction possibility comes true in ….ty years..

        • user

          call your mom. she needs to give you the meds again.

          how do you think controllers are tracked when the headset uses inside out?

          • Sponge Bob

            what is “inside-out” for controllers ???

            p.s. don’t need any meds – just need a beer or two to soften up :)

          • user

            if you only track the controllers with cameras on the headset you can never get high quality tracking because of the large area where the controllers are occluded. thats why you would need inside out for controllers too. there is an older tango demo where the players, who are in the same room, wear headsets with inside out and they pass around a tango tablet, which has inside out tracking, too. i hope i remember it correctly.

          • Sponge Bob

            “large area where the controllers are occluded” ?

            occluded by what ?

            you mean insufficient FoV of cameras ?

            Some headsets have 4 cameras – enough to see in all directions

            if you mean “can be occluded by each other for some period of time” then any tracking system like HTC of Oculus has the same issue

            btw, VR for gamers is gonna be different from VR for general consumers (and cost 5-10 times more)

            Like special computer mouse for gamers is a wired model which sells for like 100$, and I don’t remember when was the last time that I used wired mouse (10 years ago ?)

          • user

            occluded by your own body! turn your head with your 4 cameras to the left and see where your previously tracked right hand controller will go.

          • Sponge Bob

            then more cameras ???

            2 on each side :)

            plus 4 up front makes it 8

            still cheaper than your si-fi :):):)

          • Get Schwifty!

            I suspect the controller question is going to be some kind of communication system that correlates the controller relative to the headset position. Perhaps the controllers have their own positional system, maybe some kind of wireless tracking transmitted through the headset and correlated to the rest of the tracking. I too have wondered what they will do here, camera based systems have to “look” at the user with controllers from outside… be nice if someone would at least comment on if even supporting controllers is part of the plan.

          • Sponge Bob

            Plan ??? :):):)

            From reading this blog alone I can clearly sense that no one has any global plan at all – every little (and big) company is pushing their little precious over-hyped piece to make a buck asap

            “…the controllers have their own positional system, maybe some kind of wireless tracking transmitted”

            yeah, right

            maybe “some kind of wireless tracking” :):):)

          • Get Schwifty!

            LOL Maybe they don’t feel the need to share all the details yet? Since you have all the answers, please, tell us how it will be done…

          • David

            No you moron. A lack of details is usually an indicator of no-support. It’s called marketing.

          • Sponge Bob

            My prediction for the next 3 years:

            VR tech will radically split (it already did) into practical usable mobile (or “almost” mobile) VR (“mobile” does not mean smart phone-based – those will be gone from VR space soon enough- completely wrong form factor and power/heat dissipation reqs) for 95% of us normal users (not crazy gamers) and high-end stationary installations for those 5% of really dedicated gamer population willing to spend $$$$ and have dedicated “VR room” in their house
            I just fail to see where this “inside-out” tech from Eonite belongs:
            it’s obviously meant for mobile VR…. BUT no controller tracking
            PLUS high-end tablet grade GPU/CPU required just for head tracking – lots of $$$ and heat
            For high-end VR no one needs it either – Lighthouse is cheaper and already does a better job :-)

          • dogtato

            Yeah, I took the time to read the article, but I was really just looking for mention of controllers. Solving the cost and processing power problems is really cool, but it’s not really solved until it can fully replace outside-in.

      • maiskorn123

        That’s exactly why I suspect inside-out tracking will only be truly useful for AR or mobile VR, because those likely won’t ever be very reliant on external input-devices and rather focus on alternative control methods like gesture recognition.
        I like to think Lighthouse based tracking will establish itself as the prominent tracking tech for home-use consumer VR.

        • Sponge Bob

          mobile VR doesn’t need high-quality interaction ???
          what gesture recognition and by what ?

        • Get Schwifty!

          I think the biggest issue Lighthouse will have is scaling and utter lack of environmental awareness. It’s impractical to stick light sensors on everything, hence the draw for camera based systems. They aren’t perfect, but they are improving constantly and allow for a lot more information to be utilized than just the Lighthouse style. I’ll say it again, I think Lighthouse is great for what it does, but I suspect it is an interim technology.

          • Sponge Bob

            FYI

            high-speed camera-based motion tracking is a commodity tech employed by many movie studios etc etc. etc..
            those lighthouse sensors are not sensors – they are real-time fpga-based compute modules plus custom low-latency RF transmitters – not cheap at all

          • Get Schwifty!

            LOL I’m not referring to just the cameras, but the whole software suites driving them and ways the are used. Camera tech including the software to process it is continuing to improve, its not “tapped out” by any means. Just think of facial imaging systems in airports for instance.

            The Lighthouse system depends upon compute modules that are not “sensors”, yes I know that :)

            The point is use of those modules is not a practical system if your intention is to pull in the environment on any scale. And the price will come down, as with all tech, but its at the end of the day only good (very good) for close to exact positioning in a relatively small area. Yes, you can add Lighthouse units, but it’s not very practical on a larger scale. To do that, you need multiple cameras that can process imaging from a larger area and correlate it.

    • Armando Tavares

      There is no ‘IF’ about “inside-out tracking” working or not. It DOES work: http://www.roadtovr.com/qualcomms-new-mobile-tech-unlocks-impressive-inside-tracking/

      As you can read in that news, there is nothing ‘magical’ or gimmicky about it :)

      All you can say at present time is that it isn’t perfect but we are witnessing the birth of the technology. Should we expect perfection at this point?

      What you say is true: “…IF works, will instantly eliminate HTC and Oculus tracking devices altogether, and tons of other companies too..” that’s why you see all the negativity and nay saying surrounding these news.

      It’s frigin mind boggling how anything about “inside-out tracking”, which btw is AMAZING news, draw in so much ‘hate’ about them.

      • Sponge Bob

        dude

        i’m not in this frigging VR business (not yet) so am pretty neutral

        DO yo uthink Oculus and HTC arew that stupud to invest in camera or laser based tracking solutions if this “inside-out” magic is just around the corner ?

        Do you think it can ever beat camera or laser based solutions ?

        Why is it even practical for dedicated VR setups ?

        its meant for mobile VR but it will only work inside of small rooms

  • NooYawker

    I’m sure it works but so did that Xbox kincet. It’s works, but not very well. Small companies with big promises is becoming too common.
    Ideally this inside out would be best but I’m skeptical at this point.

  • We need an industry-wide standardized procedure/benchmark to evaluate 6DOF tracking claims.

    • Sponge Bob

      yeah, right

      they won;t even allow your to see their datastream, not even after merging with IMU data

      someone need to hack the rf link to htc vive controller to at leats get merged data

      HackRF anyone ?

      p.s. check FCC filings – there must be a public doc defining RF frequencies used bu Vive controllers and headset

      i just dont have time for this crap

      • I’m not talking about an internal benchmark, that wouldn’t be practical considering each tech can have a different approach.

        I’m talking about a test that an independent, dedicated, enthusiast could perform to reproduce and evaluate the device if he’s access to it. This would consider the hardware as a blackbox and use an apparatus to produce a trackable event with known ground truth.

        For example if you manage to film the headset externally and also film the through its lens at the same time with synchronized high speed cameras, you would be able to analyze the difference between headset motion and virtual world motion.

        For tracking AR markers there are simpler procedures that we could implement. I like the sine-fitting method.

        Ref: Measuring Latency in Virtual Environments http://wp.cs.ucl.ac.uk/sebastianfriston/wp-content/uploads/sites/3/2014/09/Friston-Steed-2014-Measuring-Latency-in-Virtual-Environments.pdf

        • Sponge Bob

          filming headset is no good

          it has to be on moving and precisely controlled robot’s-arm(or head:) synced with video stream from VR

          but for this inside-out thing that’ not enough either – you have to reproduce the entire environment – the whole room with all objects ad stuff and painted walls and pictures on walls etc etc

          lots of fun

  • DaKangaroo

    In my opinion, this would be the best way to enjoy VR:
    – Dedicated PC for VR in the living room
    – Don’t own a coffee table
    – Wireless VR headset using the new 60GHz wifi standard. 4 displays (like Panasonics new headset), with 2 high res panels for frontal vision and lower resolution panels for outer vision, with a fused lens, for very high FOV, and scattered LED lights outside of that for peripheral vision. Built in headphones of course
    – Room tracking via IMUs, depth sensors and image recognition to use visually unique objects in a room for tracking for the headset. Can use depth sensors to automatically build a map of the room and compute a ‘play area’.
    – Wearable shin leg pads, and wearable gloves (that go from fingers to forearm) for hand and feet tracking. Tracking hands and feet accurately provides plenty of options for locomotion options in VR environment. Combined with head tracking and IK, full body can be mostly tracked pretty accurately.

    So lets say you come home from work/school/uni/whatever, walk in the door, drop your bag and change into some casual clothes. You’re looking to relax and do some VR. What do you need to do?

    Just walk into the living room, switch the TV to the VR machine, press the button to turn on your dedicated VR machine. While it boots up, put on the pair of gloves and shin pads. Machine boots directly into VR mode 60 seconds later. You wear the headset when it says it’s ready, fold down the headphones, and you’re in VR. Use your hands and feet to interact with the world, look around inside of it, use a VR menu to select a game to play, etc.

    Then maybe pick a game, like Fallout 5, use pretend walking style locomotion to explore the world, use your hands to interact with things, picking them up, crafting stuff, shooting guns, etc.

    How awesome would that be?

    We *almost* have all the right tech to make that a reality.

    • Sponge Bob

      Dude, you don’t have a clue

      If you have a dedicated VR room in your house then why the hell do you need inside-out tracking ???? It is absolutely useless in this scenario

      what’s wrong with installing lighthouses or cameras at each room corner ?

      And how are you gonna track your gloves and pads ? those will have inside-out tracking too ? LOL

  • Eelke Folmer

    Positional tracking is great but it still doesn’t solve the VR locomotion problem. Unless it lets you move through physical infrastructure, navigation will always be constrained to available tracking space. It looks like this solution is powered by a battery in a backpack, because powering a 3D sensor using your smartphone will drain its battery in 15 minutes. This approach really needs a different solution.

    • Sponge Bob

      dude

      its not a battery – its a high-end tablet-grade gpu/cpu-processing unit… with a battery and a lot of heat coming out…

      not for phones… for the next 3-5 years

      • beestee
        • Sponge Bob

          impressive

          lets hope it won’t explode

          I say smart-phone form-factor is completely wrong for VR power and heat dissipation requirements

          iIs just volumes are not high enough to drop the prices for high-quality standalone mobile VR headsets to say a few hundred bucks so they use ill-fitted expensive smart-phones for this purpose.
          will change very soon

          • beestee

            “lets hope it won’t explode”

            All bets are off the table there, sorry. Effects of ignoring Moore’s Law?

          • Sponge Bob

            There are also thermodynamics laws – those are real Laws of Nature (unlike Moore’s “law”)

            I say: Smart phone form factor is completely wrong for mobile VR
            Why ? – Power and heat dissipation requirements

    • user

      what about all the tango phones? i doubt they will only last for 15 minutes.

  • Armando Tavares

    Naysayers everywhere… lol

    If I didn’t know any better I’d say you naysayers are rooting for anything that INS’T Oculus/Vive related, to fail. But I’d be wrong… right?

    I just hope some deep pockets company (like Facebook) doesn’t buy this in order to bury it.

    • Sponge Bob

      dude, its not some miracle hardware

      its software company doing algorithmic refinement of the imperfect depth sensor data

      how can you bury published articles and phd thesis ?

      • Armando Tavares

        Easy: You buy it and then do nothing with it.

        Maybe it wouldn’t kill the technology but it could set it back enough to prevent ‘some’ company from losing tens of millions.

        • Sponge Bob

          dude

          you just dont understand how tech works

          you cant even hide patents once they are published – even thoufghg those are :private:” priperty and you can be sued for using them without license

          once something is out in public then everybody and his brother cries me tooo and starts copying like crazy, patented or not

          buying phd thesis and and preventing companies from developing on it’s ideas ??? :):):)

          I wish someone bought mine years ago lol

          • Armando Tavares

            I’ll try explaining what I mean with a little example: What do you think would happen to the electric car (as a concept/product) if Ford (for example) buys TESLA, fires everyone and then does nothing with the company or it’s patents? It wouldn’t destroy the electric car (as a concept) but I bet it would set it back, maybe, tens of years.

            That’s how tech and patents work.

          • Sponge Bob

            No dude

            patents are just published docs and they are used without permission all the time, especially in high-tech, so buying up patents and not using them in products does nothing to stop the progress by other companies – especially in other countries)

            unless tesla keeps some deep trade secret on how to make those batteries there is nothinbg to prevent japanese from making same freaking tesla cars – they will even look the same but will be called hondas or fudjis or whatever :)

            I know this from first-hand (and very unfortunate) experience. LOL

  • Tipatat is involved. I’m sold!

    • Sponge Bob

      whf is tipatat ?

      better tell us if we should already through our lighthouses and oculus cameras in the garbage can or not yet ?

  • Albert Hartman

    This is really only for VR, not AR. AR can just use crappy room trackers for the generated overlaid graphics – the rest of the room views just fine. This is really