NVIDIA announced on Thursday the release of its CloudXR 4.0 platform at the GPU Technology Conference (GTC) 2023.
The new tools will empower developers with bespoke CloudXR use cases for applications and clients. It will also provide extended reality (XR) scaling for deployments across the cloud and improved server and client flexibility for developing applications.
With the upgrades, developers can tap the platform for use in the cloud, over 5G mobile compute edge (MEC), and corporate and enterprise networks.
It also introduces fresh application programme interfaces (APIs) that can provide developers with multiple tools for connecting and interfacing with CloudXR. These include analytics insights on network performance, upscaling on multiple platforms, and streaming optimisation for high-capacity networks.
CloudXR 4.0 also facilitates work with Unity-based architecture via a sample client. The new tools will allow developers to create bespoke interfaces with a single client operating on multiple XR devices.
Some of the top takeaways for CloudXR 4.0 include:
- CloudXR Server API, offering developers CloudXR integrations directly into apps. This removes the need for server-side OpenVR and OpenXR runtime. NVIDIA is also working with Collabora to incorporate Monado OpenXR as CloudXR’s API. It will continue to support OpenVR API on CloudXR using the SteamVR runtime.
- Unity Plugin, which seamlessly integrates Unity XR API and others on CloudXR. This provides developers with the capacity to build fully-featured CloudXR clients via the Unity Engine, deployable across XR client platforms.
- Low Latency Low Loss Scalable Throughput (L4S), offering ‘togglable’ advanced 5G packet delivery optimisation. This dramatically reduces video streaming lag across cloud-based servers.
The novel solution also replaces physical tethering and boosts high-fidelity XR streaming for mobile devices. NVIDIA’s latest toolkit also allows clients to serve customers across devices.
Comments on NVIDIA CloudXR 4.0
Speaking on CloudXR 4.0, Matt Coppinger, Director of Product Management, VMware said his company used it to “enable our customers to stream high-fidelity XR experiences. These included VMware’s Horizon XR platform and standalone devices using VMware’s Workspace ONE XR Hub.
He added,
“This gives our customers the power of a graphics workstation along with the mobility of a standalone VR device. The flexibility [that] the CloudXR 4.0 release provides with new application and client APIs will allow us to improve the CloudXR integration with Workspace ONE XR Hub and improve overall solution performance”
Additionally, NVIDIA partnered with telecom firms such as Ericsson and Deutsche Telekom to implement the former’s L4S solution on CloudXR. This provided the most optimal streaming platform for high-bandwidth, low-latency networks, the company explained.
In a statement, Dominik Schnieders, Head of Edge Computing, Deutsche Telekom, explained that his company had seen “great optimization” with L4S and CloudXR, which reduced “latency and network jitter.”
He added: “This will be a critical component of streaming XR for both enterprises and to customers in the public 5G network.
Additional integrations and features for APIs and 5G networks include:
- Generic Controller Support, allowing CloudXR client developers to map inputs for their head-mounted display (HMD) controllers.
- Call Back-Based Logging for enhanced flexibility in handling log messages. This applies where applications can determine callback functions and utilise log messages to perform tasks such as writing to file.
- Flexible Stream Creation to determine which streams it should launch for devices or platforms.
Additionally, CloudXR 4.0 joins Project Aurora to develop a “full hardware and software stack” for distributing XR solutions across public and private networks. The innovation would become the “linchpin” for added flexibility, ease of deployment, and performance, allowing IT departments to roll out XR applications across networks.
NVIDIA’s Greg Jones on CloudXR 4.0
XR Today spoke to Greg Jones, Director of Global Business Development and Product Management, NVIDIA, about its latest CloudXR update.
When asked why NVIDIA’s CloudXR platform was critical to building interoperability, Jones said his firm saw a “fundamental trend” in head-mounted displays (HMDs).
New devices moved to “lighter-weight, more streamlined profiles” while boosting resolution, frame update rates, and field of view (FoV).
He explained further,
“For graphics processing, these lightweight devices may be underpowered and under-resourced relative to memory for the graphics workloads that support experiences users desire. This is especially true in enterprise use cases where the digital models tend to be very large and is likely true for consumer use cases where photorealism is an important driver”
Many AR and MR use cases would enhance operations with AI, requiring more computing power than available locally on lightweight mobile devices, he added.
Supporting XR Streaming for Enterprise and 5G Edge Networks
Jones explained that to supply memory and compute power for increasing graphics and AI workloads, devices needed GPUs to remote stream to mobile devices.
Regarding streaming capabilities, Jones said that HMD manufacturers initially offered streaming options from local PCs, which typically worked well.
However, generalised systems such as the Cloud, corporate networks, and 5G edge networks proved difficult to ensure interoperability when devices reached “out to these services with unique streaming architectures.”
He told XR Today,
“This is where platforms like NVIDIA CloudXR can sit on remote servers, offering a common API for application developers to write to and a common API for HMDs to accept for remote streaming”
Such APIs, namely on servers, would quickly change to “support scalability over the next few years.”
Jones concluded this would take place as XR ecosystems moved from distributing applications to “users with specialized desktop systems to streaming to a broader user base defined by individuals with simply an XR device.”
Growing Number of Use Cases for Unity API Integration
XR Today also asked about use cases expected to leverage NVIDIA’s Unity API integration for CloudXR. Jones said the team was “really excited” about the tool, which would provide “two major advantages” for developers.
Firstly, it would create a universal client “for all HMD devices that support OpenXR,” Jones explained. The plugin would also “allow developers to create user interfaces using Unity, [which is] seeing greater adoption, especially in the mobile application arena.”
He explained further,
“This ability is the number one request we hear from our developer community. We see the creation of application-specific experiences where the developer uses Unity to build experiences and branding of user interaction on the HMD, say in a lobby, before the user launches the actual immersive experience running on a remote platform. The immersive experience running remotely can be written in anything, such as a custom renderer like Autodesk VRED, ESI Group’s IC.IDO, or Unreal or Unity engines.”
Partnerships with Global Telecoms
XR Today later asked how NVIDIA’s partnerships with Deutsche Telekom, British Telecom (BT), VMware, Ericsson, and others facilitated technological development essential to CloudXR.
Jones said such partnerships were “the essential ingredients to building a scalable, interoperable platform for immersive experiences.”
Continuing, he explained,
“Making sure state-of-the-art optimizations like L4S are in the streaming architecture means operators are assured that, when they offer an XR application via their 5G network, that application will offer a great experience using all the state-of-the-art optimizations available”
He added that NVIDIA had been working with Deutsche Telekom and Ericsson to optimise XR delivery over 5G for roughly three years. Collaborations first began in laboratory settings, then moved to private networks and later, optimisations “into the standards groups so public networks can utilize them.”
Concluding, he said: “This is Ericsson’s bread and butter, so partnering with them to move XR through this process to create an interoperable platform, optimized for high capacity, low latency networks, makes a ton of sense.”
The NVIDIA GTC event is taking place from 20 to 23 and showcases some of the Santa Clara-based firm’s latest technological advancements.
The conference featured keynote speeches from Jensen Huang, Founder and Chief Executive, NVIDIA, along with numerous use cases for generative AI, NVIDIA Omniverse, and digital twins, among others.
