Digital twins are an extensive piece of the XR toolbelt. The technology is unique in its flexibility and usability across different devices and sectors.

Commonly, enterprise end-users leverage digital twins to replicate real-life objects or environments. This allows decision-makers to work on immersive visualizations as part of their daily operations, such as reviewing iterations of a product or viewing a digitized urban area to simulate smart city initiatives.

Moreover, digital twins are an incredibly valuable tool for training scenarios. A workplace trainee can learn how to use or repair a piece of equipment using a digital twin; some solutions allow trainees to take apart complex digital objects, like an engineer, to see the inner component – saving a lot of time, usually reserved for clean-up.

Digital twins, in all use cases, can reduce spending. Instead of having to physically bring an object to a person or bring a person to the object, digital twins allow decision-makers to review digitized items from anywhere in the world, speeding up processes and helping with environmental concerns.

Moreover, digital twins are truly an interoperable technology, meaning workers can view and interact with them using a range of hardware, from smartphones to headsets.

However, the technology is still finding its place in the enterprise. While some use cases are more significant in popularity, there is still room for digital twins to integrate into other sectors and further into the future of work.

Joining today is:

• Max Bickley, Senior Product Designer of Omniverse at NVIDIA
• Tim Giuliani, Orlando Economic Partnership President & CEO
• Betsy Gilbert, Marketing and Business Development at Arvizio 
• Omer Shamay, Treedis CEO

How do digital twins operate in tandem with XR hardware and software?

Max Bickley

Digital twins operate in tandem with XR hardware and software by leveraging immersive visualization, real-time data integration, remote collaboration, and simulation capabilities. In the future, AI will play an increasingly significant role in assisting our utilization of digital twins.

Visualization: Fundamentally, XR displays allow users to visualize digital twins in immersive environments. By wearing these devices, users can interact with at-scale virtual replicas of physical objects or systems, enabling them to inspect, manipulate, and explore the digital twin representation.

Data Integration: XR service providers integrate real-time data from digital twins into XR experiences.

By connecting sensors and IoT devices, including sensors in the XR device, to the digital twin, XR applications can visualize and overlay real-time data and AI-based data analysis onto the physical objects observed through XR devices. This integration enhances situational awareness and decision-making by providing contextually relevant information.

Remote Collaboration: XR facilitates remote collaboration among teams. Multiple users wearing XR devices can access and interact with digital twins simultaneously, regardless of their physical location.

This capability allows for remote training, remote assistance, and collaborative problem-solving, as users can share a common virtual space and interact with the digital twin together.

Simulation and Training: XR technology enables realistic simulations based on digital twins. Users can practice tasks, procedures, or scenarios in a virtual environment that replicates the behaviour of the physical system.

By integrating with digital twins, XR provides a dynamic and interactive training platform, enhancing skills development and knowledge transfer.

Betsy Gilbert

Digital twins, when paired with headsets and mobile devices, provide organizations with the ability to view and interact with 3D models, whether as a stand-alone representation or in conjunction with a real-world physical product or object.

Many industries are implementing XR technology, utilizing a combination of hardware and software, to enhance and improve their equipment service, training, and knowledge transfer initiatives.

The ability to project comprehensive step-by-step instructions with interactive visual cues directly onto the work surface is an innovative and effective method to guide workers and direct their attention to the relevant aspects of complex tasks one step at a time.

This ensures that their focus is on the quality of that one action, substantially reducing time and errors while improving productivity and knowledge retention. The incorporation of real-time remote expert collaboration with analytical shared features ensures that critical operator actions are accurate.

Arvizio AR Instructor is designed for headsets and mobile devices, incorporates digital twins, other media such as documentation and photos, and symbolic assets to allow customers to create customized digital instructional scenarios. In addition, we provide the ability to connect with remote experts for additional guidance or work validation. 

Omer Shamay 

At Treedis, we have witnessed the seamless integration of digital twins with XR hardware and software.

By leveraging XR technologies such as AR and VR, users can interact with virtual representations of physical assets or environments.

Digital twins serve as the foundation, providing the underlying data and context to create immersive experiences.

XR service and hardware providers today integrate digital twins by using them as the backbone for visualizing and manipulating virtual objects in real-time, enhancing user engagement and understanding. 

How can digital twins help us to improve decision-making? 

Max Bickley

Today, customers across industries are digitizing their industrial datasets, processes, and systems. These digitized assets and systems can be used to create digital representations of physical assets and environments, such as plants and facilities.

The industrial metaverse is the next evolution of digital transformation, describing the concept of full-physical-fidelity, real-time, immersive digital twins and industrial virtual worlds that are perfectly synchronized and obey the laws of physics.

These virtual worlds require several pieces of technology: a physically accurate, large-scale virtual world simulation engine such as NVIDIA Omniverse; the ability to automate and achieve superhuman abilities to optimize, using AI; and interoperability between industrial design and production ecosystems.

We now have the technology to create true-to-reality digital twins of the physical world, and this new evolution of the web will be much larger than the physical world because, like the web, almost every industry will benefit from participating and hosting virtual worlds.

Creators will make more things for virtual worlds than they do in the physical world, and enterprises will build countless digital twins of products, environments, and spaces — from object to planetary scale.

Simulation brings enormous opportunities for all enterprises, such as testing thousands of different product designs and scenarios virtually before physical production, saving on cost and waste, and increasing operational efficiency and accuracy.

Tim Giuliani

The Orlando Regional Digital Twin was designed and built to do exactly that—improve the ability for viewers and users to make informed decisions. 

The Orlando Economic Partnership has learned that when a company is making a relocation or expansion decision, its planners typically visit three to 10 cities. And they only spend a few days to a few hours in each location to understand its value for business expansion. 

Our digital twin, which maps 800 square miles of the Orlando region, helps site selectors and corporate CEOs see the breadth of developable land, available real estate, areas of interest for a variety of industries, infrastructure connectivity, and talent availability.

Orlando’s digital twin could be viewed as a more in-depth version of Google Maps that allows decision-makers to experience a “flyover” of the region with overlays of relevant data points. They can also view heat maps with demographic information, transit routes, rail spurs, and fibre optic cable connections. 

The digital twin helps executives, consultants, site selectors, and community partners better understand the Orlando region without having to spend multiple days in a car, allowing them greater ease with which to make their decision. 

We also see the digital twin as a foundation for a platform that can serve as a critical resource for decision makers in the area and play a vital role in Orlando’s economic future and smart city planning. For example, stakeholders could use the digital twin to simulate climate change mitigation projects based on historical rainfall totals and flooding patterns.

Decision makers could predict traffic bottlenecks using sensor data from stoplights and highways, informing transportation planning. City planners could use historical population data to predict future density for use in transit mapping, housing development, and power grid expansion. 

Betsy Gilbert

Digital twins can play an important role in fostering an increase in decision intelligence. The digital twin is a 3D digital representation of the physical object capable of improving decision-making throughout the asset’s lifecycle.

The use of digital twins in the design phase is well-known: the digital asset offers the opportunity to assess, change and optimize the product prior to production.

To complete the lifecycle, the optimized digital twin design can then be used to develop and refine installation, operation, and maintenance instructions – providing solutions to predicted issues and repairs. This will drive convergence and understanding across the involved workforce.

By incorporating digital twins into training simulation programs such as AR Instructor, organizations will reduce the learning curve for upskilling and new employees and vastly improve the decisions made while performing their tasks.

The use of digital twins that operate across the entirety of the asset lifecycle allows effective decisions to be made that align product management, reliability, strategic asset investment, and workforce training and planning. 

Omer Shamay 

Digital twins have proven instrumental in enhancing decision-making processes.

With our experience at Treedis, we have witnessed how digital twins provide real-time insights and predictive capabilities.

By simulating different scenarios and analyzing data collected from digital twins, organizations can make data-driven decisions, optimize operations, and predict outcomes.

The ability to monitor and analyze the performance of physical assets in real-time empowers decision-makers to identify potential issues, take proactive measures, and optimize resources, leading to improved efficiency and cost reduction.

What role does AI play within the digital twin landscape?

Max Bickley

The first wave of AI is about digital intelligence running in data centres. The next will be about the software increasingly coming out of computers and data centres and running in the world, bringing automation to physical things such as plants, buildings, and factories.  

To develop the software to do that, we need to develop virtual worlds where AIs can learn and be put through testing before being released into production. 

This is why digital twins are so important and why they need to be physically based and accurate representations of physical industrial systems. 

Today, many companies think about their physical products. As they increasingly design, test, and configure for optimal performance, all virtually, they’ll begin to think about the importance of the digital representation of their products.

These digital twins will become as important to them as their physical counterparts. ‌These AI-enabled twins will bring a level of acceleration and capabilities they’ve been dreaming of — allowing them to directly predict and optimize their physical products.

So far, AI has only really been used for the light industry. For it to be used at scale for complex industrial digitalization (healthcare, manufacturing, automotive, semiconductor, etc.), the industries need their operations to be represented digitally.   

AI today is largely used for text-to-text or text-to-image. In the future, text-to-animation will be possible. To know, for example, that a robot’s motion is grounded in reality and physics, software systems need to understand the laws of physics. 

Factories of the future will be digital first. Each will have a digital counterpart. Everything that moves will have AI and robotic capability and be designed, simulated, and trained inside physically accurate virtual worlds and digital twins. 

Tim Giuliani

Orlando’s digital twin currently incorporates operational and sensory data.

Future versions will incorporate AI and machine learning to allow for scenario mapping, modelling, and simulation, empowering users to run what-if scenarios and understand how new developments, initiatives and programs will impact the region well-before implementation.

Betsy Gilbert

AI, used in conjunction with digital twins, can provide significant improvement in object recognition, error detection and failure warning, assembly and various automation and test simulations resulting in both time and cost savings.

The greatest challenge with AI implementation is the vast number of high-quality training datasets needed to learn and recognize items, identify issues, and recommend resolutions.

Digital twin models can be used to accelerate the training by generating the datasets and simulating faults and anomalies.

Arvizio is incorporating AI into our AR Instructor to provide an augmented and mixed reality solution that enables workers and technicians to look at a physical asset (via an XR headset or tablet) and receive immediate identifying information, functionality, and faults.

Utilized, for instance, in information technology or manufacturing, the combination of technologies creates an intelligent operation by providing timely fault warnings, predicting device health, and processing conditions to reduce field maintenance travel and extend equipment service life. 

Omer Shamay 

This is something we’ve experienced first-hand, as our whole platform is based on AI and its ability to convert 2D images to 3D space (Matterport Digital Twin), creating a digital twin that you can use through Treedis. 

By leveraging AI, digital twins can also provide intelligent insights, automate decision-making processes, and enable predictive analytics, customer-based personalization, and more…

AI-powered digital twins continuously learn and adapt, driving optimization, efficiency, and innovation. 

How should vendors and end-users manage digital twins to ensure accuracy, privacy, and security?

Betsy Gilbert

Digital Twins become most useful when managed by a cloud-based solution allowing important data to be accessed from any location.

This brings with it security and access control concerns. For example, training and instructional workflows may be used by a diverse audience, but only specific users should have authoring rights to change the content.

A robust access control and management framework is required to organize and manage the digital twin assets.

Omer Shamay 

At Treedis, we prioritize accuracy, privacy, and security in managing digital twins.

We implement robust data governance practices to ensure the accuracy and integrity of the data used in digital twins.

Access control mechanisms are put in place to safeguard sensitive information and restrict unauthorized access. Encryption techniques are employed to protect data transmission and storage.

Regular updates and maintenance are performed to address vulnerabilities and ensure the security of the digital twin infrastructure.

Compliance with regulations and standards, such as GDPR, is a fundamental aspect of our approach to protect privacy and data security. 

What types of organizations are adopting digital twins are scale, and what other markets can the technology expand into?

Max Bickley

Enterprises across all industries are developing digital twins.

For example, BMW is using the NVIDIA Omniverse platform to build digital twins of its factories to optimize its production line ahead of new vehicle launches.

Siemens Gamesa simulates wind farms to maximize energy yield.

Ericsson uses Omniverse to create digital twins of cities to decide where to place 5G towers and virtually fix problems before deploying crews on the ground.  

Here are a few more examples: 

• Amazon has over 200 robotics facilities that handle millions of packages each day. It’s a complex operation that requires over half a million mobile drive robots to support warehouse logistics. Using Omniverse and Isaac Sim, Amazon Robotics is building AI-enabled digital twins of its warehouses to better optimize warehouse design and flow and train more intelligent robotic solutions. 
• Lowe’s is using Omniverse to simulate everything from where nails are placed on store shelves to how aisles should be reconfigured for the holiday rush. Lowe’s is aiming to create hyper-personalized and hyper-localized shopping experiences for its customers in the U.S., Canada, and Mexico. The retailer has implemented a first-of-its-kind use case for Omniverse at two test stores in Charlotte, North Carolina, and Seattle, Washington. 
• Digital Rail Germany, or Digitale Schiene Deutschland, part of Germany’s national railway operator Deutsche Bahn, is building the first country-scale digital twin to fully simulate automatic train operations across an entire network in collaboration with NVIDIA. 

Thousands of companies around the world use NVIDIA AI to develop and deploy models for a broad range of applications, including recommender systems, customer service and digital assistants, factory safety, supply chain optimization, and more. 

Tim Giuliani

In the Orlando region, we’ve been following digital twin development by innovative companies like Oracle, Siemens, and BRIDG (Bridging the Innovation Development Gap) — a public-private partnership bringing advanced microelectronics research to market through the fabrication of silicon wafers. These businesses are using the technology in a variety of ways, including prototyping, simulating new processes, and modelling performance based on real-world variables. 

Additionally, The VA’s Simulation Learning, Evaluation, Assessment and Research Network (SimLEARN), a program based in Orlando for ideating and measuring simulation technology in healthcare training, is creating a digital twin application to enhance best practices for the VA’s facility designs. The application will use AI-driven modelling for rapidly iterative simulations of real-life scenarios to expand the testing of possible solutions to a given challenge. 

For the Orlando Regional Digital Twin, we chose to work with San Francisco, California, USA-based Unity Technologies as the lead technology partner for the project.

Unity had experience building digital twins for other organizations, including Tyndall Air Force Base, which used the technology to streamline the design, planning, operations, and sustainment of the installation’s rebuilding after Hurricane Michael.

Using enterprise data systems and building information model (BIM) updates, the digital twin supports operations and seeks to improve the safety, resilience, and performance of the base.

Looking ahead, our hope is that more Orlando organizations leverage the Orlando Regional Digital Twin to improve smart city planning. For example, stakeholders could use the digital twin to simulate climate change mitigation projects based on historical rainfall totals and flooding patterns.

Decision-makers could predict traffic bottlenecks using sensor data from stoplights and highways, informing transportation planning. City planners could use historical population data to predict future density for use in transit mapping, housing development and power grid expansion. 

Soon there will be future uses for digital twins and similar technologies that we can’t even fathom today. There are use cases that haven’t even been imagined yet. 

Betsy Gilbert

From our perspective, enterprises across industries are engaged with the use of digital twins for both on and offsite operational and training scenarios.

This includes manufacturing, production, and field service in sectors such as Information Technology, Manufacturing, Aerospace, Automotive, Energy, and Government, where accuracy is paramount.

However, we see a natural expansion of the use of digital twins in Education where institutions can train, guide, and validate students as they learn how to install, operate, and repair complex equipment.

The use of digital twins as accurate representations of physical objects ensures that both training and functional activities are consistent across the organization. 

Omer Shamay 

We have witnessed various organizations adopting digital twins at scale, particularly in the manufacturing, energy, and construction sectors.

However, the applications of digital twins are not limited to these industries. Digital twins have the potential to expand into other markets, such as smart cities, retail, agriculture, logistics, and infrastructure management.

By leveraging the power of digital twins, organizations can optimize processes, improve decision-making, and drive innovation across diverse sectors, transforming the way they operate and deliver value.