AI Is A Critical Building Block For Spatial Computing

We are at the cusp of a new computing paradigm. It’s called spatial computing. Spatial computing is when physical and virtual objects converge seamlessly and it's made possible with artificial intelligence. This has profound implications for how we interact with technology and with each other. Spatial computing opens up endless possibilities for creativity, innovation, human connectivity, and new ways to work. It removes barriers, closes distances, and enables co-presence. Spatial Computing will make the devices we use and how we use them blend into the daily natural flow and patterns of how we live our lives.

To appreciate the business value of Spatial Computing, we first have to create a working definition for the business world and explain the market opportunities it will enable. Once we unlock that, we can understand how business and computing will change in order to prepare for this transformation.

As I wrote in the Harvard Business Review, “Spatial computing is an evolving form of computing that blends our physical world and virtual experiences using a wide range of technologies, thus enabling humans to interact and communicate in new ways with each other and with machines, as well as giving machines the capabilities to navigate and understand our physical environment in new ways. From a business perspective… it will expand computing into everything you can see, touch, and know.”

Spatial Computing is an evolving 3D-centric form of computing that, at its core, uses AI, Computer Vision, and extended reality to blend virtual experiences into the physical world that breaks free from screens and makes all surfaces spatial interfaces. It allows humans, devices, computers, robots, and virtual beings to navigate via computing in 3D space. It ushers in a new paradigm for human-to-human interaction as well as human-computer interaction. These new interactions will enhance how we visualize, simulate, and interact with data in physical or virtual locations. It will expand computing beyond the confines of the screen into everything you can see, experience, and know.

Spatial Computing allows us to navigate the world alongside robots, drones, cars, virtual assistants, and beyond. It’s not limited to just one technology or device. It is a mix of software, hardware, and information that allows humans and technology to connect in new ways ushering in a new form of computing that could be even more impactful than personal computing and mobile computing have been to society.

Spatial computing brings digital information and experiences into a physical environment. It takes into account the position, orientation, and context of the wearer, as well as the objects and surfaces around it. It uses a new, advanced type of computing to understand the physical world in relation to virtual environments and the wearer. It does this by using emerging interfaces like wearable headsets that have cameras, scanners, microphones, and other sensors built into the device. New interfaces come in the form of shopping to work and play. The world around us will talk to us in new ways via spatial computing. Spatial computing enables advanced gestural recognition (like recognizing our hand motions and applying them as commands) and it will have better-than-4K resolution images for each eye.

Spatial computing uses information about the environment around it to act in a way that’s most intuitive for the person using it. How businesses digitally transform using spatial computing will set them apart from the competition and set them up for success for generations who grow up in an increasingly blended virtual and physical world.

Spatial Computing Is Not Just About AR And VR

To many, spatial computing may not seem different from virtual reality or augmented reality. Augmented reality is overlaying digital content into a physical space. Virtual reality is a completely immersive virtual environment. The extended reality (XR) spectrum is part of spatial computing, but it's not its only enabling technology. Artificial intelligence is on everyone’s mind, along with XR, sensors, IoT, and new levels of connectivity. AI is one of the most important underlying technologies that will bring spatial computing to the masses.

Spatial computing is a mix of hardware and software that enables machines to understand our physical environment without us telling it so. That enables us to create content, products, and services that have a purpose in both physical and virtual environments.

The future of spatial computing is poised for substantial growth, driven by key advancements. These include radical progress in optics, the miniaturization of sensors and chips, the ability to authentically portray 3D images, and the continuous evolution of spatial computing hardware and software. These innovations, supported by significant breakthroughs in AI, will make spatial computing increasingly compelling for businesses on a grand scale in the years to come.

AI’s Critical Role In Spatial Computing

Artificial intelligence has a critical role in spatial computing. There are applications personal to the wearer and objects in the spatial environment. We won’t type to AI. We’ll talk to it. AI capabilities like large language models (LLMs) and deep neural networks (DNNs) allow people to act and think like humans as they interact with computers (instead of thinking like a database). Amazon’s Alexa Smart glasses say it all: let “voice control your world - hands-free.” AI’s critical role in spatial computing will force business professionals to start thinking about spatial computing in the context of today’s AI revolution.

Here are some of the ways AI is a critical building block for spatial computing.

AI recognizes our hand gestures and body language.

AI algorithms, like computer vision models, improve the accuracy and speed of object recognition and tracking. This is crucial for understanding our hand gestures to interact with the digital environment. AI can also identify and track objects and surfaces, which makes spatial computing experiences seamless and interactive. Spatial computing uses cameras, LiDAR, GPS, and sensors to capture the position, orientation, and movement of objects in an environment. Spatial computing allows us to place and manipulate virtual objects in ways that correspond to the virtual world.

Object and scene generation.

People will use spatial computing and AI together to generate content for digital spaces. Just like content creators today use their cell phones to create online content, so too will they use their wearable spatial devices to create 3D content. From games to spatial video and audio, creators will use AI to develop personalized scenes, objects, and entertainment for their fans.

Conversational navigation.

Human beings are social animals. We thrive around other humans. While some argue that no one would wear a device on their faces, similarly, many doubted we would ever use the internet or mobile phones. Spatial computing will solve communication challenges as simple as where to find a product at your favorite store or share your child’s birthday party with relatives far away. There is a sense of volumetric presence. It’s more than the sense of someone being there on a video call. It’s as if they’re sitting next to you. A 3D hologram you could reach out and touch,

Spatial computing will also serve as the medium to engage with AI.

AI takes data and generates new data. For DeepMind and Inflection AI founder Mustafa Suleyman, the next phase of artificial intelligence will be interactive AI. That begs the question of how and where we will engage with this interactive AI. That’s where spatial computing comes in. Spatial computing has become the medium through which we engage with AI in a more human way. We will navigate the future of computing most of the time either through gestural recognition (which uses AI) or through conversational AI. While we might still type on virtual keyboards, it might not be convenient for simple day-to-day tasks.

Some foreshadowing of this change was seen during Apple’s Apple Event, where the new iPhone 15 and new Apple Watch were unveiled. Apple is conditioning the market to use neural interfaces to engage with technology through a natural double-tap motion on the Apple Watch.

This was a big step for Amazon as well. They recently announced their new Alexa LLM and Echo frames rooted in style and in a more robust conversational AI interface. Amazon pushes the hands-free narrative by talking to Alexa to make calls, play an audiobook, or playlist. Talk to your Amazon glasses to check that the doors are locked, lights are set, and even adjust the thermostat with your voice.

Spatial navigation for autonomous vehicles, drones, and robots.

Spatial computing isn’t just for humans. Objects like autonomous vehicles, drones, and robots need to know how to navigate the physical world. Spatial computing, along with AI, helps robots “see” and interpret the world around them. Whether it’s knowing when to stop or go at a stoplight with pedestrians or orienting itself through a manufacturing facility, spatial computing with AI will guide robots through our world.

In some ways, the spatial computer hardware becomes the interface through which humans are able to interact with AI in a more human-centered way, away from the screen and bringing it into our physical world. Through spatial computing, we will engage and see AI in the form of delivery robots, autonomous vehicles, drones, humanoid robots, or virtual beings and assistants navigating the world around us.

Spatial Computing Is A Key Step in The AI-Driven Business Revolution

Through AI, IoT, sensors, and more, spatial computing enables the creation of larger connected ecosystems that seamlessly integrate virtualization and data in many forms with our physical world and in front of our eyes, no matter where we are. In some ways, it helps elevate or augment the way we experience the physical world. This, in turn, will create new experiences, new ways to do business, and new utilities that can make spatial computing a valuable tool for innovation and business transformation.