The Apple Vision Pro has made sensational headlines across global media outlets, leading to renewed interest in virtual, augmented, mixed, and extended reality (VR/AR/MR/XR) technologies.
Further clamour for the long-awaited headset arrived after the Cupertino-based firm announced on Monday it had opened applications to develop apps and creative content for the premium device.
The Apple Developer Program will allow qualified developers across the globe to receive firsthand experience in creating apps, interfaces, and real-time 3D (RT3D) content. Doing so will also provide greater interoperability with additional platforms such as Unity and Xcode, among others.
In a recent Big News episode, five professionals from the XR industry shared their insights on the implications of such content. This ranged from user interfaces (UIs), filming, controls, and hardware optimisation.
For the Big News round table discussion, XR Today spoke to:
- Kevin O’Donovan, Co-Chair, VRARA Industrial Metaverse and Digital Twin Committee
- Jay Latta, CEO and Founder, STINT
- Florian Carls, Chapter President Munich, VRARA
- David Ware, Film Production Professional
- Rory Greener, Technology Journalist, XR Today
The group of experts also leveraged their expertise to determine the device’s trajectory as a ‘breakthrough’ mixed reality (MR) headset. Further debate on the strengths and weaknesses of the celebrated headset assessed its long-term efficiency and viability.
Need for Empirical Evidence with ‘Vision Pro’
Speaking on his thoughts about the Apple Vision Pro, Latta explained that commentators needed more empirical experience with the device.
Citing informal social media polls ahead of the Worldwide Developers Conference (WWDC 2023), Latta stated that many had resorted to “clickbait,” rather than trialling the headset. Due to the immense secrecy of the Vision Pro, many had to wait until after the big reveal at the annual event.
“Everyone was even waiting till Apple announced something. It is not sure what they are building in their garage,” he said.
Despite this, Latta praised the headset’s features like its field of view (FoV), controller-less hand tracking, eye and gaze tracking, and 3D video recording capabilities. This could lead to new forms of interacting with sports events.
He said: “I might use some algorithms to bring them to life, which is simply a completely new quality. This is something a small feature that would be very, very interesting.”
From Strength to Strength
Ware chimed in that a key takeaway of the device was that, despite its current $3,500 USD price point, competition would grow massively in “a couple of generations’ time.”
He said that the Vision Pro had reinvigorated global interest in XR, adding that collaboration and competition with the headsets were “all about the future.”
Ware explained further,
“Everyone’s working together making their standpoints towards the future goals and headsets. I read earlier someone talking about how Apple’s focusing on sexy and another said, ‘well, for $3,000 its had better be incredibly sexy.’ However, in reality, none of the headsets are sexy. You’re essentially walking around with giant devices stuck on your head, not that dissimilar from where it was 20 years ago. However, I guarantee that, 10 years from now, you’re going to have [immersive] sunglasses or eyewear like XR contact lenses, which people have already been working on”
The industry needed incremental progress towards future form factors and advanced hardware. Additionally, devices required vital content to power immersive experiences, along with software platforms to complement them.
Apple Vision Pro: A Big Win for Spatial Computing
O’Donovan added that Apple deserved “huge credit” for reframing conversations around XR. He noted that Apple spoke less about XR but rather about spatial computing.
For him, concepts such as spatial computing, consumer usage, and Vision Pro features would face tests in the real world. “Now we need to see the stuff working,” he said.
He said,
“If they can perfect their new R1 chip running, reach zero-latency, and controllerless interfacing working in real-time, this changes the game. We can do that today with headsets, but there’s lag, particularly with networks. If the Vision Pro can live up to the technical specs, then this thing is very interesting, but they still need to deliver it”
Comparatively, the Vision Pro’s price tag matched that of the Microsoft HoloLens and Magic Leap 2.
“It’s great that they’ve changed the goalposts. We’ll see how it plays out, but they’ve got to deliver what it is,” he added.
Apple Vision Pro News: No ‘Breakthrough’ Yet for XR
Highlighting the device further, Carls explained that the device was “not a breakthrough yet” but was a “huge step in the right direction and had a lot of potential.
The hand and eye-tracking expert explained that the Vision Pro could serve as a “spatial computing platform that we can build on,” but was not yet social.
He added that families were unlikely to buy the device and watch Disney programmes together. However, he expected the headset to facilitate this more after a few generations.
Carls concluded: “We have seen these kinds of experiences on other headsets before, so the price is the main limiting issue due to its hardware.”
Controller-less Headsets: The Future of XR
When asked about Apple’s approach to controller-less human-computer interfacing (HCI) and its long-term viability for enterprise use cases, Carls said it was a “great way forward.”
He said that headsets without controllers were a “great way forward,” allowing developers to consider natural interactions such as hand, eye, and other movements.
Carls added devices and programmes to maximise the use of immersive spaces, “especially in enterprise setups.”
He explained further,
“Looking at the production line, I don’t want to have controllers in my hand or to look for them. I really want to grab and interact with digital objects as naturally as possible, which is a really big step”
Microsoft IVAS Criticisms: Gestures and Military Sign Language
Citing issues with his previous startup, Sprint Active, he said that the platform did not feature controllers. Interactions with others took place in the shared spaces, allowing for fully natural interactions.
He said at the time,
“Additionally, the whole UI for controllerless interaction needs tochange. We do not need floating sticky notes or virtual keyboards. The UI needs to be more like a magician that you open up, have all of your items that you simply choose from, especially after the HoloLens [Integrated Visual Augmentation System] disaster in the defence sector. This was the hint that certain interfaces do not work for all kinds of jobs. You develop a specific sign language while working and interacting with coworkers in the military sector”
He noted that it was even more difficult to communicate in military operations with such mixed reality devices, creating confusion for other soldiers.
Carls said: “The other guys may be looking at you like, ‘What do you want me to do? Do you want me to proceed?’ This is the level of smart application these devices need to arrive at.”
Adding to the conversation, O’Donovan stated that, should Apple reduce the latency of the device to an “imperceptible” level, devices could deliver accurate gesture controls and improve device functionality.
He continued,
“Without controllers, you open the space to a lot of people who are not used to having controllers as well. It’s really the mass adoption for all ages, who can use the headset with gestures more than control. I think it’s a smart move.”
Eye Tracking and the New Movigoer Experience
Ware added that, at a recent screening of his VR short film, there were guests of all ages. Many “had no idea how to control [the headsets], even some of the younger generations,” he said.
Continuing, Ware added,
“What we’re talking about here is Minority Report-like technology, where you’re interfacing with AR screens in front of you. I would also agree that a keyboard is not a good idea. I think that developers will have a difficult time getting people to understand how to create their own shortcuts again, even developers and people that are more technologically advanced”
He noted, however, that the device depended on the target audience. Mass consumer audiences would require designers and developers to pick UIs that were more universal in nature.
Ware said: “The idea is to get to the level of Jarvis technology from Ironman, where it’s a combination of AI and eye tracking technology, similar to the ones that are actually starting to take place.”
Regarding films, Ware explained they were “passive mediums” that did not require audience participation. However, with VR filmmaking, people looking away and paying attention to other items could receive feedback from characters.
Digital characters could then prompt the character to look at an object or respond, based on the eye-tracking technologies from the device.
Ware concluded,
“Now, that’s amazing if that arrived next year. If you can do that with the interface, then perhaps engineers, game engine designer, or other professional, you would have to consider the [human] senses. There are a million ways to explore [this technology], and its just going to take time to experiment with, figuring out the best ones and actually making it happen”
Latta joined in, stating that these kinds of interactions began with volumetric movies, where viewers could move around the character or imagine the film from the director’s perspective.
For more information, please follow XR Today’s Big News Show
