This Tuesday, Nreal unveiled a hand-tracking update to its augmented reality (AR) software development kit (SDK) for the Chinese company’s lineup of smart glasses, including new controller support and eye tracking options.
AI+AR = our new Hand Tracking API. Developed with AI, virtual interactions in 3D become more intuitive.
Download and learn more here:https://t.co/mdZ95KINFX pic.twitter.com/p9ODE4DVAk
— Nreal 👓 (@Nreal) August 16, 2022
The update utilises artificial intelligence (AI) systems to create accurate hand tracking and meshing. The Nreal SDK runs on the Unity-based Microsoft Mixed Reality Toolkit (MRTK), and applies to Nreal’s Air and Light smart glasses.
Core NRSDK Updates
Nreal built the hand-tracking update using two open-source MRTK applications, “Hand Tracking Hub” and “Startup Scene,” available via Microsoft Design Labs.
The applications help Nreal developers manage custom immersive environments, textures, and inputs. The SDK briefing outlines how the Design Labs’ applications are the “blueprint” to its hand-tracking features.
Additionally, the Nreal SDK update contains two example projects to introduce a developer to the MR SDK and various pre-configured assets, settings, and features, including:
- Hand interaction, including buttons, object scaling, and scrolling
- Hand skeletal meshes and gestures
- Eye tracking
- Planes detection
- Speech to text
- Hand menus (user interface – UI)
- Joystick support
- Non-native keyboard support
- pre-loaded assets like lava and lighting
- Pre-loaded objects, such as a piano
Hand-Tracking Coming Soon
Nreal introduced hand-tracking for its products last month, announcing options to control immersive adverts with hand gestures, letting users skip an advert after five seconds, continue watching, call-to-action click-throughs, and timestamp tools.
Also, the update gives brands more control over the distribution of immersive marketing campaigns with tools to spatially place AR adverts in a user’s environment. Brands like L’Oreal, Adidas, and Gucci are developing AR-ready adverts with Nreal’s hardware lineup.
Nreal debuted the Air smart glasses in the UK last May, with distribution partners EE assisting in spreading the device across the region, which is available for £395.
Nreal designed its product with a focus on media streaming. The Air device includes a 46-degree field of view, 90Hz refresh rate, 77-gram frame, and 5G integration.
The Chinese hardware vendor works with major sporting events to enhance broadcasts using immersive AR visualisations. In August, Nreal partnered with Immersiv.io and the Devon Football League to provide an AR live stream of this year’s SuperCup 2022.
Additionally, the smart glasses firm worked with BT Sport to enhance the Premier League. Nreal also worked with Orange to create 5G-powered visualisations during the Tour de France 2022.
