Oculus Research Devises High-accuracy Low-cost Stylus for Writing & Drawing in VR

11

Using a single camera and a 3D-printed dodecahedron decorated with binary square markers, the so-called ‘DodecaPen’ achieves submillimeter-accurate 6DoF tracking of a passive stylus. Lead by National Taiwan University PhD student Po-Chen Wu during his internship at Oculus Research, the work presents a low-cost tracking and input solution with many potential applications in virtual and augmented reality.

As shown in the video below, the ‘passive stylus’ in this case is an actual ball-point pen, allowing for a quick visual demonstration of the impressive accuracy of the tracking system, with the real and digitised drawings being almost indistinguishable from each other. Although the project focused on stylus applications, the video also highlights how the dodecahedron could be attached to other objects for virtual tracking, such as a physical keyboard.

According to the paper published on the NTU’s website, the DodecaPen’s absolute accuracy of 0.4mm is comparable to an advanced OptiTrack motion capture setup using 10 cameras—a combined resolution of 17 megapixels. The DodecaPen system achieves the same accuracy with a single, off-the-shelf, 1.3MP camera. The research clearly shows that marker corner alignment alone is not enough for robust tracking; the team instead used a combination of techniques detailed in the paper, including Approximate Pose Estimation and Dense Pose Refinement. The 12-sided shape was chosen to retain constant tracking quality, so that “at least two planes are visible in most cases.”

The key advantage of the DodecaPen is its simple construction and minimal electronics, making it particularly suited to 2D and 3D drawing. However, the team recognises its limitations and drawbacks, being prone occlusion due to the single camera, and relying on a reasonable amount of ambient light to maintain accuracy. Also, the paper notes that their computer vision algorithm is ‘slow’ compared to 300-800Hz motion capture systems, as well as Lumitrack, another low-cost tracking technology. DodecaPen’s solution is limited by the fiducial marker recognition software and the motion blur generated by the camera, resulting in unwanted latency.

The conclusion states that the system could “easily be augmented with buttons for discrete input and an inertial measurement unit to reduce latency and increase throughput.” A more complex stylus could also offer a better simulation of real drawing, including pressure sensitivity and tip tilt, which would make it better suited to emulate a pencil or brush rather than a pen. The problems of occlusion and limited low-light performance could be improved with multiple cameras with higher quality image sensors and lenses, but each upgrade would add to the system’s cost and complexity.

SEE ALSO
Oculus Research Reveals "Groundbreaking" Focal Surface Display

A made-for-VR stylus like the DodecaPen could prove to be a versatile tool for traditional productivity tasks in VR, which are largely limited today by a missing solution for fast and easy text input.

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.


The trial version of Microsoft’s Monster Truck Madness probably had something to do with it. And certainly the original Super Mario Kart and Gran Turismo. A car nut from an early age, Dominic was always drawn to racing games above all other genres. Now a seasoned driving simulation enthusiast, and former editor of Sim Racer magazine, Dominic has followed virtual reality developments with keen interest, as cockpit-based simulation is a perfect match for the technology. Conditions could hardly be more ideal, a scientist once said. Writing about simulators lead him to Road to VR, whose broad coverage of the industry revealed the bigger picture and limitless potential of the medium. Passionate about technology and a lifelong PC gamer, Dominic suffers from the ‘tweak for days’ PC gaming condition, where he plays the same section over and over at every possible combination of visual settings to find the right balance between fidelity and performance. Based within The Fens of Lincolnshire (it’s very flat), Dominic can sometimes be found marvelling at the real world’s ‘draw distance’, wishing virtual technologies would catch up.
  • Walextheone

    The keyboard tracking, or tracking of any object is amazing. Such a low cost tech with a big impact for usability

    • Firestorm185

      ikr? That’s pretty huge.

  • Raphael

    Can be used to draw an adorable bunny. This is what sold me.

  • Nice experimentation, but I don’t see a practical application

    • Darshan

      I see… check my post above

    • Ragbone

      Playing darts.

  • Darshan

    Hmm Great apply this now on Standalone VR headset Oculus Go..

    Nothing is more promisingly amazing than merging Windows VR Cameras +Oculus Go low cost hmd + Dodecahedron attached controllers fully tracked in VR…bring this at $250 and i can bet .. Oculus will not be able to handle demand for years to come..

    Glorious Victory for VR ahead…..

  • Ragbone

    Would be good for playing darts games.

  • JV

    I see low cost full body tracking for oculus!

  • Foreign Devil

    Happy to see progress on this! However. . without something simulating pressure sensitivity (and it could be as simple as a very pressure sensitive trigger), it wont’ be much more useful than occulus touch for sculpting and drawing.

  • Lucidfeuer

    Their proposed “6DOF” tracking algorithms is way smarter and efficient than tech porcoration were ever able to figure-out…again start-ups and academics doing the work of greedy wastes.