Cloudhead Games is one of the most successful and senior VR studios in the industry. In this Guest Article, studio head Denny Unger shares his thoughts on Apple’s entrance into the space.

Guest Article by Denny Unger

Denny Unger is CEO and CCO at Cloudhead Games. Based in British Columbia and founded in 2012 Cloudhead’s pioneering approach to VR gave rise to broadly adopted movement standards including Snap Turns and Teleportation. Working closely with Valve, Sony, and Meta, Cloudhead is best known for their title Pistol Whip and has shipped four popular VR titles (Pistol Whip, Valve’s Aperture Hand Labs, Call of the Starseed, and Heart of the Emberstone).

So let’s get the obvious over first; Apple Vision Pro is Apple’s first generation attempt at AR glasses using a Mixed Reality VR headset. AVP is a development platform also serving an enthusiast demographic. Make no mistake, this no compromise MR device appears to get many things right for AR at a premium cost. Will Cloudhead Games be buying one to better understand Apple’s approach? Heck yes. AVP will give developers a powerful foundation and ecosystem for which to develop AR apps for a future ‘glasses formfactor’ device in that mythical 5–10 year window. And to the victor, the spoils of a smartphone replacing device.

No doubt (and if rumors are true) there were many debates at Apple HQ about VR. Whether or not to open the device up to VR studios and successful titles. Whether or not to include controllers to support legacy VR titles. Whether to allow users to full-dive into Virtual Reality, freely move around, and be active in the medium. But in an effort to sharpen their messaging, and to command a dominating lead within the AR space, VR and its many benefits were expertly omitted on nearly every level. Do I understand the strategy to strike a different cord as an XR business owner? Absolutely. Does it frustrate me as a VR-centric studio owner? You bet it does.

Image courtesy Apple

I question why the AVP didn’t maximize its potential, leveraging almost a decade of know-how from the VR community working within this space. Why not set a vision for a future device that would accommodate both AR and VR as complimentary mediums? Apple could have embraced a dual launch strategy with a rich and proven catalog of best selling VR games, perfectly tuned to onboard a completely new audience to XR. Apple could have expanded into VR’s recent success, growth and competition within the current market. In their recent presentation VR is essentially reduced to a gimmick, the thing you lightly touch the edges of, instead of a complimentary and equally important medium. Unity engine support is promised but with no plans for motion control support, Apple has cut out any possibility of porting most of the existing or future VR catalog to its platform.

Hand-tracking is a logical affordance for AR based spatial computing and no doubt some experiences will work well with that design philosophy. However it is important to point out that most VR games built over the last 10 years (and many more in production) are not compatible with, nor will they ever be “portable” to hand-tracking only design. Inputs and Haptics are incredibly important to Virtual Reality as a major tenant in reinforcing immersion and tactile interaction with virtual objects. Buttons pushed, triggers pulled, vibrational feedback experienced, objects held, thrown or touched, alternative movement schemes supported. There is a comfort in understanding the topological landscape of a controller and a physical touchpoint within the virtual environments themselves. When introducing users to a radically different medium like VR, convention & feedback matters. And over the last 50 years in gaming, input has evolved to encourage a suite of highly refined game design standards, creating a particular kind of muscle memory in the gaming population. Say what you will about which genres remain popular in this 450 Billion dollar industry but it does strain belief to think we’ll all be playing with finger guns in the latest and greatest shooter.

SEE ALSO
Vision Pro Teardown Shows Balancing Act Between Cutting Edge Tech & Weighty Design

I know what some are likely to say “ there will be new innovative standards and we’ll look back on controllers as a crutch”, but I would push back and say hand-tracked or not, moving away from future haptic devices and innovation is a backwards step in XR design. Even smartphone games utilize basic haptics, because touch is foundational to the human experience.

In the aftermath of the AVP launch some would argue that VR is not yet mainstream and that Apple did the right thing by ignoring it. I would argue that VR turned a significant mainstream corner when Quest 2 outsold Xbox, when Sony reentered the market with PSVR2, and when Google teamed up with Samsung to work on what’s next, and on it goes. Over its 10 year rebirth, the last 3 years of VR have experienced Hockey Stick levels of growth. OEM’s have increased investments, and significant indicators keep coming with more titles earning revenues north of $20 Million. Fully immersive VR is a legitimized medium not because I say it is but because people like it, and are willing to part with their hard earned money to experience it.

Image courtesy Apple

I hope Apple is more inclusive of VR over time but the Apple Vision Pro appears to be a VR headset pretending not to be a VR headset. Because of this strategy it represents a unique opportunity for Apple’s competitors to double-down on supporting Virtual Reality at a more affordable entry point. Sure, they can all wage the 5-10 year war for a smartphone replacement but why in the world would one ignore an equally compelling revenue stream within a blended MR ecosystem? Maybe, because it took too long to go mainstream? Sorry all, we had to learn a few things along the way but I’m happy to say that after 10 years, the trail ahead has never been this clear.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Road to VR
  • impurekind

    Totally hear you on that.

    • Dragon Marble

      People assume this was a bold move from Apple. Maybe they ignored VR because they are afraid to directly complete with Meta on the same lane.

      • Will

        They ignored it in their marketing. That doesn’t mean they’ve actually ignored it. They’re trying to make a breakthrough, mainstream device. In doing so, they chose wisely to focus on it as an AR device that keeps you connected to the world and people around you. Their “spatial computing” branding is a brilliant way to emphasize this, as is the fact that when you put it on, the first thing you see is passthrough of the room you’re sitting in. The physical dial that tunes between AR and VR is the icing on top.

        None of this changes the fact that it is still a killer VR headset that’s actually using VR to simulate an AR experience. I think it’s very shortsighted to assume they won’t bring in VR after they’ve got everyone confortable with the ecosystem. It’s still a big market, it just doesn’t represent the breakthrough they’re looking for. In the same WWDC, they unveiled a new translation layer for the M chips that literally lets them run every Windows app. All they need to do is release controllers, or even just allow 3rd party controllers, and the entire Steam VR ecosystem is right there.

  • Carla

    Maybe they ignored it because it’s a sick dystopian future where shooting anything that moves for fun creates a society of psychopaths, a certain number that turn into mass killers!

    • Kfir Even

      Are you living in 1993 still? we’ve moved on from this debate and apparently shooting games didn’t turn us all into psychopaths

    • Nevets

      This is such a silly comment that I consider you a likely troll.

    • polysix

      Hi pops, nice to hear from you again. I’d argue that people addicted to shooting games are too busy playing shooting games to actually bother tooling up and going out to kill real people.

      The people who do THAT are extremely disturbed individuals, nothing to do with videogames… did Mark Chapman shoot Lennon cos of GTA… or cos of a Book (more.. cos of his own INSANITY), whatever they ‘blame’ it on at the time – music, books, TV, movies… games… no…

      Videogames are not the problem.. GUNS are.

      • Guest

        Letting off steam is not as strong as monkey see, monkey do! That combined with lots of guns is why there’s a school shooting almost every day in America. The people making and selling all these gun games will reap their karma someday!

  • another juan

    apple always has been kinda embarrassed of gaming. at least you can say they still stay true to their principles

    • Christian Schildwaechter

      During the WWDC keynote that ended with the introduction of the AVP, Apple also presented their Game Porting Toolkit, a set of mostly translation libraries to assist developers with porting existing PC games to Mac. Officially intended for studios to without time-consuming tests get a quick impression of how well the ported games might perform, by running Windows games on a Mac with live translation, the porting toolkit was pretty much immediately used by some gamers to run everything from Cyberpunk 2077 to Elden Ring on M1/M2 Macs.

      There is obviously a frame rate penalty for needing to translate at runtime both the graphics API from DX 11/12 to Metal (similar to Proton on the Steam Deck translating to Vulkan), but also the x86 code to ARM instructions via the Rosetta 2 tool they already had before. The result is about 50% performance loss compared to the same game available in native form. But since there are very few graphically demanding games with native Mac versions, and the M1 and M2 are pretty fast, this opens up the gigantic PC gaming catalogue to Macs, at a decent quality, for free.

      Apple may have been “kinda embarrassed of gaming”, and their engagement with iOS gaming may be mostly due to the ginormous pile of money it generates for them. But occasionally they break with their principles and give gamers what they desperately want, even if instead of declaring the Mac compatible to most Windows games like Valve did with the Steam Deck, they release a tool officially only to be used by game devs to test the feasibility of native ports, that then “accidentally” makes porting unnecessary for many cases and can be used by everyone.

  • fcpw

    Anyone who wants to make haptics controllers can do so, they already support anything bluetooth. Does Apple really have to do everything itself to satisfy this guy?

    • Wishbringer

      Well, most VR games rely on specially designed (and tracked) motion controllers, not standard bluetooth gamepads. Relying on optional 3rd party solutions probably wont win a lot of developer support. So yeah, I think Apple does have to do it.

    • Andrew Jakobs

      If Apple would support OpenXR then it wouldn’t be a big problem, but having to implement every bluetooth controller yourself is not fun.

    • Christian Schildwaechter

      As already pointed out, gamepad support is not the same as 6DoF controller support. Developers couldn’t even hack in their own support for e.g. the self tracking Quest Pro controllers, if Apple doesn’t allow to override the default hand tracking input. At best they could use Immersive/VR mode, ignore the Apple input system, implement a completely new input system and way to render the users hands and arms. And I’m not sure if you can even disable the automatic rendering of an avatar based on hand tracking by the OS itself, so you might end up with two overlapping pairs of hands and arm (though I guess you can). Acceptable for an experiment, but for all practical purposes Apple effectively prevents using 6DoF controllers.

  • Till Eulenspiegel

    The endgame is an AR glasses – but current tech is not good enough to create one.

    The only solution today is to use a video feed and superimposed AR element by using tracking – hence the need for the bulky VR ‘clothes’. It’s basically AR done in VR.

    Eventually the tech will gravitate towards miniaturization into a glasses. That’s the goal.

  • XRC

    Great article, have alot of respect for cloudhead, from early days of Vive roomscale with Call of the Starseed to more recently pistol Whip.

    As someone obsessed with hand input design, motion controllers are in their infancy; there are considerable improvements yet to be made to input components, haptics, ergonomic adjustment and durability / repairability.

    Hand tracking for everyday spatial computing makes sense, but for gaming I’d expect to see more specialized equipment to develop, akin to flight simulators. Part of my openXRC controller development includes game specific controller components to enhance input or feedback.

  • I can’t question Denny, he’s just too amazing :) . But I think that Apple is just being Apple and taking its decisions… for now it launched this way proposing it as a luxury item, but let’s see what creators will ship and what people will like.

  • Mark

    For me VR/AR is about sitting relaxed experiences, I don’t want to come home after work and jump around in the latest VR gun game. I mean how long do people stay in with physical games? An hour at most? AVP points to longer relaxed experiences. As it is the only game I play ATM is demo because I can’t be bothered with the physical stuff. This is in part why flat screen games are so popular because humans are naturaly lazy, we want the most fun for the least effort and current VR games dont give that but AVP just might.

    • Andrew Jakobs

      I can do 4 hours in a ‘roomscale’ VR game, and most of the time I just stop because it’s getting late, not because I’m getting tired or something. AR is something I really don’t have any interest in, and sitting still on my couch with my headset on is also not something for me.

    • JanO

      This is exactly why I’m afraid of the mainstream… It dumbs down everything it touches. There are already plenty of games that fit the lazy. VR has only opened the door for a new kind of games I really enjoy; games for active and healthy gamers! This is way better than the current fatso lazy couch gamer paradygm, which can still exist in that same space.

      Sitting down has become the most widespread human desease, just as dopamine has become the universal drug big corporations feed our minds with for maximized profit.

      • Mark

        We are not all fat and lazy, just tired. I am a postie and I bike 5 miles a day to and from work and I walk 80 miles a week with heavy bags. There are many many people with physical jobs that are bone weary at the end of the day but enjoy easy VR experiences. ;)

        • Dragon Marble

          That’s impressive! I thought active VR would be most appealing to people like you. It seems that if you simply walk 70 instead of 80 miles, it would spare you enough energy for lots of fun + same health benefits!

    • NotMikeD

      I think there are two types of gamers; one type wants to feel like they’re going on an adventure and the more immersive (even physical) the experience the better.

      The other likes to sit back, swill Mountain Dew and pop Cheetos, and rack up the hours in a recliner or swivel chair.

      I think it’s important to have options that cater to both. I don’t necessarily think it’s a bad thing for the physical stuff even IF what you’re claiming is true and folks only put in an hour at a time; so what, a sale’s a sale. I don’t NEED to wring 1,000 hours out of every $30-$60 game I buy as if every video game was a Destiny or a Minecraft. I’m a dad, 1-2 hours at a time is all I’ll ever get anyway, I’d personally rather spend that hour doing something immersive and physical, something that actually makes me FEEL something, than grinding out another soulless RNG grind.

    • polysix

      I often play resident evil VR games – standing/turning… (first on PSVR2 now on Quest Pro wirelessly which is much better for that) for hours – usually only stopping to charge something…

      I went through the night for 6 hours on PSVR2 once, standing…

      MANY VR games just don’t feel right sitting, nor as immersive… esp roaming/walking/shooting games.

      so I guess you’re a lazy ****

      Of course, there are times when seated VR is perfect/only option (GT7… Star Wars Squadrons etc), it’s good to have the variety but you should mimic whatever the game is trying to convey to get proper VR.. not sit down if you’re standing/moving/aiming in game… for that you have TV games.

      • Mark

        so I guess you’re a lazy ****

        Absolutley not, see post above.

        • silvaring

          Hey Mark. What’s happening here is that some people want to pick a fight with you, because they are trying to protect their narrow concepts of what they think VR should be. Tribalism exists here perhaps its the power of the VR experiences these individuals have had, and the fact they don’t try other forms of entertainment much anymore. I recall someone in a thread a few days ago talking about how all they want in VR is to be able to enjoy their HOTAS setup… I mean if that’s your only goal, a very niche, limited experience then good for you, but for people to make it seem like a equipment device manufacturer should change their whole system to cater to your limited niche is so hilarious.

          And right now VR as a whole is so incredibly niche, but even worse it has a staleness attached to it, and its going to be the casual market that turns that around, not some niche group of people playing a flight sim in their custom desk setup.

    • This is a thoughtful take, sorry you’re getting trashed in the comments. The crux of the issue is why go out of your way to exclude active VR content from AVP?
      Think about it this way, a lot of people struggle with maintaining adequate activity levels in their lives. VR and fitness is a really interesting mix. You get all the fun engaging mechanics and story telling of traditional video games while also burning some extra calories.
      I’ve certainly been more active since having VR gaming in my life and it’s something I value from VR.

    • Nepenthe

      An hour is about how long I would ever have to play ANY game in any orientation. VR is great to me because it enables something so different from couch sitting.

  • philingreat

    In my opinion, Apple doesn’t want any experience where you either move around a lot or that has even the slightest potential to make you motion sick. They want that every single person has a relaxed and good experience when going into AR/VR. Maybe that’s needed to make it mainstream. Not with the AVP, but the Apple Vision that comes as a next iteration.

    • polysix

      the problem with this… is that it’s not compelling for a long time, retention is low if you’re NOT embracing actual VR (with wireless, Controllers, TOP PC graphics, haptics etc)… this has been shown over and over… people buy in then don’t use cos not compelled to.. and the more dumbed down you make it (i.e sit on sofa waving hands around with very little to actually do or reason to be in vs watching TV/reading a book/tablet for the net) people wll see it as a fad and put it next to the wii… excercise bike etc in short time (i’m talking mass market casuals).

      It’s still people ‘not getting VR’ and trying to shoehorn it into less compelling/easily digestable formats for the dumb masses when all along REAL VR is, for a long long time, only going to be extremely desirable to true VR nuts like me/us… VR itself needs to get better and better (esp graphically) then hands/haptics, then locomotion… this currently can only be done ‘properly’ via PC (wireless with Quest Pro is my choice after 7 ex HMDs… inc recent psvr2 which was a bust in the end – mura – sweet spot – cable – underpowered ps5 )

      Apple just ‘want in’ at this early stage but don’t actually ‘care’ about VR like META do.. my money is still on META to actually push VR properly, and on Valve to push it for hardcore PCVR.

      • philingreat

        No one would have expected that a phone with no controllers would create the most successful gaming platform in existence. But it happened because everyone could play, all the time.

        • Luka

          Mobile gaming is a success because you can play anytime, anywhere. That’s why we also play “all the time”. With Apple kit or any VR, you can’t play on a sidewalk, ina bus, while cooking, or waiting in a doctor’s office, during a boring meeting, etc. I mean, technically you could, but both the awkwardness and the 2h battery will put an end to it like immediately. That’s applicable to working on phone or browsing on it vs VR/AR/XR as well. Until the AR is built into our eyes directly somewhere down the line. So comparing mobile gaming success to AR isn’t a good comparison. Phones were a specific revolution that managed to put a computer (and Internet) into everyone’s pocket, and it will take decade(s) or more to get to technical breakthrough that will allow an AR experience that trumps cell phones, and be available for – everyone.

    • Mark

      I prefer this kind of experience and I am of the opinion this is why VR hasnt gone mainstream, I dont know why people think everyone wants to be on their feet getting active. Humans want the most fun for the least effort, reward vs effort is foremost in the minds of every animal on the planet, this is a fact.

  • alxslr

    Tim Cook just never liked VR. Period.
    Developers will have to find the way through this “someone’s personal preference” handicap, and push for VR experiences for what is indeed a VR headset. And for the controllers, even though I find shooters will have real problems to survive without some kind of device, I believe exploratory games similar to “The Gallery” could still thrive.

  • jasonmartino

    VR has had 10 years to go mainstream and it hasn’t. Why should Apple double down on a business model that doesn’t work? The demos are very non-active, people sitting on a couch or working. They didn’t even tie it into fitness apps, only meditation. The last thing Apple wants is someone throwing up. That would make the AVP dead on arrival. My guess is that if they do go after a more active experience, it will start with fitness way before VR games.

    • Christian Schildwaechter

      Apple is going for games, if only for the money. They never really pushed gaming as a use case for their devices, it mostly happened because the user wanted games, so Apple provided the required support for developers. They now are in third place after Tencent and Sony regarding revenue from gaming. And in first place regarding profits from gaming with a significant lead on everybody else, making more money than Sony, Microsoft, Nintendo and Activision combined.

      So they won’t miss that opportunity on AVP, and they won’t limit it to playing iOS games via a bluetooth gamepad or selling their Apple Arcade game subscription service. Just like with the iPhone, they will force developers to play by Apple’s rules, so no controllers, a lot of demands regarding comfort and more, requiring developers to come up with new types of games. Which those will, as creating iPhone games that rely mostly on taps and swipes also made them a lot of money.

      • jasonmartino

        Good points. Didn’t realize Apple made so much money on Scoops. Adventure games with eye-controlled teleport, console VR games with gamepad support like Tetris and Moss, puzzles, job simulators, mini golf, beat saber clones, boxing, fishing, tabletop games, Real Time Strategy, and painting should satisfy the mainstream. The list above will not make people sick. I would advise Apple (at first) to discourage games with smooth locomotion, rails, flight sims, FP racing, horror, and guns – that’s Quest and PCVR’s domain – and once a non-gamer gets sick, they are likely never to come back.

        • Christian Schildwaechter

          They benefit from having both a large and affluent install base. Android has a much larger install base, but Android users are much less willing to pay for any services, so a lot of games that are released for pay on iOS are released ad based on Android. Customers with more money to spend also drive up earnings from DLC or micro-transactions, so the fact that Apple is a company focusing on high price products indirectly also drives their high earnings in gaming.

          That calculation is a problem for Meta, who have a small XR install base, even if it is the largest in VR, and at the same time their main product was deliberately designed to be cheaper than existing headsets to draw in new users. But consequently that user group didn’t spent enough on the Quest store to allow Meta to make a profit on the combined hardware and software sales like on consoles, the average Quest user bought only about half as many games as the average PS4 user, and at lower average prices.

  • Christian Schildwaechter

    VR and its many benefits were expertly omitted on nearly every level.

    That’s not entirely correct. What actually happened is that
    VR controllers and their many benefits were expertly omitted on nearly every level.

    Which is not the same. VR games already using hand tracking or easily adapted to it will port without issues, as will older mobile VR titles from e.g. GearVR that were gaze controlled and thus already in a similar way to the Vision Pro. Titans of Space, Walkabout Minigolf and Puzzling places should work just fine in AVP Immersive mode.

    Of course many don’t see these as true VR games, instead asking for titles that clearly benefit from or absolutely need controllers, which is the majority of existing titles. These titles were designed that way because 6DoF controllers was the most advanced (or only) available input type. Hand tracking on Quest exists and is usable for a lot of things, but not a proper replacement for Touch controllers.

    That’s similar to flat first person shooters being developed and designed that way because pointing and shooting at distant objects works well with a mouse, and running and jumping in four directions at one or two speed levels works well with a keyboard. Very popular existing games working well with existing input methods doesn’t mean that these are necessarily the best input methods, only that these were the available input methods and that game design was guided by the (limits of the) available input and tried to make the best of it.

    Non-gamers usually don’t like controllers at all, and controllers are obviously in the way while doing anything other than playing games. And the added input feedback in the APV of not only lighting up when close to the finger, but then also moving down and emitting a ring of light plus audio feedback when “pressed” tricked the brains of at least some of the people who got to try it enough to later say that they “felt” the click, despite knowing this wasn’t possible.

    Sure, they can all wage the 5-10 year war for a smartphone replacement but why in the world would one ignore an equally compelling revenue stream within a blended MR ecosystem?

    Maybe because they don’t see it as an equally compelling revenue stream? I could say a lot about Quest 2 outselling the Xbox Series S (during a shortage of Xbox Series X and PS5 that people actually wanted), and what “being mainstream” means or requires, but a bigger issue than sales numbers is the more obvious problem of ongoing lack of acceptance for VR within their well known and quantifiable user/gamer groups.

    PCVR usage on Steam remains below 2.5%, and it gets even worse if you also include platforms like Epic or EA. PSVR 1 never got above 5% of the PS4 base. Meta sold almost 20mn Quest, but doesn’t tell us how many are (still) used. There should be almost 10x as many Quest out there as pure PCVR HMDs ever sold, yet for games like Green Hell VR that are available on both platforms and don’t scream “mobile” at the first look, sales/ratings ratios are usually around 4:1.

    Calculating in that way is always dangerous, but we are most likely looking at a total active users number of (significantly) less than 20mn for mobile plus PC plus console VR, with estimates usually being much lower. For 2023 Statista estimates 6.9bn smartphone “subscriptions” (active contracts incl. Prepaid, less actual users due to some with multiple devices, automated services etc.), already generating 50% of all gaming revenue. And at the current rate of adding ~250mn per year, there will be more smartphone “subscriptions” than humans by 2028. So a <20mn business opportunity might be considered less compelling than one potentially 400x as big.

    • Guest

      Yeah, Apple will probably get all the GearVR devs that were abandoned by Oculus because they are scared that Samsung will get shafted yet again by another similar partner.

  • hellohellohellobye

    Explain

  • Seems plausible to me that Apple deliberately chose to avoid in-depth discussion of VR functionality primarily due to timing. AVP is still nearly a year from being commercially available. While there is no significant competition in the AR space, making it relatively safe for them to basically give everyone a big preview of all of their clever software and UI-based innovations for AR, if they tipped their hand with regards to their VR plans now, Meta (and others) would have the better part of a year to get started on playing catch-up.

    They will have another opportunity to give us a big update on AVP in October and could relatively easily announce support for 3rd-party controllers, or controllers of their own, at pretty much any point between now and launch. They had a lot of ground to cover and the decision to focus on AR allowed them to show off the stuff that is both most exciting/ground-breaking and relevant to mainstream audiences.

    Give it 6-8 months. My guess is that there will be plenty of good VR gaming content for this device (even if it is largely ported from other ones) at (or shortly after) its release.

  • Cl

    Controllers aren’t going anywhere until we get gloves with force feedback and touch. Even then, you can say that’s a controller.