Speaking during a session at GDC 2018 last month, Richard Forster, Senior Team Lead at Sony’s Research & Development West group, chronicled a range of explorations in advanced livestreaming and sharing features for PSVR.

Today PSVR users can access the usual PS4 broadcasting tools. Using the SHARE button, you can capture screenshots, videos, and livestream gameplay showing your view from the ‘Social Screen’, the cropped first-person view shown on the TV as you play. But PC VR users have had a leg up thanks to access to tools which allow for things like a picture-in-picture view of the player, and even mixed reality views where games allow players to composite themselves into the action.

In the session, Forster said that the livestreaming experiments he was going to share were not yet part of the company’s SDK roadmap for PSVR, though the session made it clear that Sony is interested in finding new and improved ways for users to share their PSVR gaming sessions with audiences.

Forster identified several different modalities which the experimental tools are positioned toward.

The first is what he called “easy access broadcasting,” the kind of low-production livestreaming that a single user could manage in their own. The approach is similar to what PSVR users can do today with the SHARE function, except this mode could allow the player to position a virtual camera within their game world, rather than only broadcasting the first-person Social Screen view. Forster suggested that the first-person view could be modified for a better sharing experiencing, including the ability to re-render the view for proper fullscreen output (rather than relying on a cropped and distorted version of the view rendered for the headset). A steadicam mode, which would smooth out shaky head movements could also be applied, in addition to post effects to change the look and feel of the output.

SEE ALSO
'Arizona Sunshine 2' Review – Head-popping Fun With Friends

Another modality, which involves a bit more production, would be to have a ‘producer on the couch’, a second user who would act as the livestream director. The second user could be sitting on the couch next to the VR player, and use a controller to adjust camera views and make other adjustments in real-time, potentially also offering on-the-spot commentary.

Forster explained that the R&D West team had built an internal Air Hockey VR demo to play with some of these ideas and see what else could be done to enhance the streaming experience.

As they began experimenting with third-person camera views, it became clear that tweaking the way the player is rendered to the third-person camera would make for a better live streaming experience. For instance, while the game alone wouldn’t have required it, Forster suggested that adding some kinematics to the avatar to make them move more realistically would make the third-person perspective look better to viewers. Fake eye-movements, like blinking and object tracking, were also something that he noted could make avatars look more believable and interesting for viewers.

Image courtesy Sony

The team also experimented with enabling some of the cool mixed reality composite mixed reality views that are popular for high-production broadcasts of PC VR content. Beyond just compositing a subject into the game using a green screen, Forster said it would also be possible for the game to output a mask which could be used to make players appropriately appear behind content in the game world, rather than just being plastered on top of it (as seen in the image above, where the player is ‘on top’ of the air hockey table instead of behind it).

Of course, compositing and masking take additional resources and could impact the performance of the VR game. Forster suggested that a second PS4 Pro could be employed which would be dedicated to generating output for real-time broadcasting and compositing, which could potentially include 4K and HDR output, additional shader effects and more.

Image courtesy Sony

In addition to sharing gameplay, Forster said that the team was motivated to explore these advanced sharing functions to help developers with promotion of their games, including the ability to output production-ready livestreams for professional streamers, and helping users capture cool moments in VR games to share online. Trade shows too, he said, would benefit from these advanced functions, giving Sony and others better ways to show off what players are seeing inside the headset while other players wait in line to try it for themselves. He also said that the tools could be helpful for the production of VR trailers, giving developers better ways to show potential players what it’s like to play the PSVR title.

Sony was showing off a number of these experiments on the show floor at GDC 2018 using various internal demos, but haven’t committed to which might reach the public.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • Jean-Sebastien Perron

    PSVR much like Vive made poor hardware decisions in the past and they are forever stuck with them. So they designed themselves for quick success without any plan for the future. Those horrible sticks will forever haunt them.

    • jj

      forever only last till next generation which have already started to roll out.

    • Sandy Wich

      The motion controls for PSVR were really cool back when they first launched on the ps3. Even though they had serious limitations… But now that they are on the PSVR… It’s just… Sad.

      Luckily though people like VR so much that there will undoubtedly be a PSVR2, and I hope they upgrade their tech because it’s sorely in need.

      PSVR atm is little better than cell phone VR.

      • Graham

        Yes you’re so right – psvr is EXACTLY like cell phone vr – I’ll just go play Skyrim vr and re7 vr on my phone. Oh hang on – it seems I can’t and that you are in fact talking rubbish.

  • Jean-Sebastien Moron

    PSVR much like Rift made poor hardware decisions in the past and they
    are forever stuck with them. So they designed themselves for quick
    success without any plan for the future. Those horrible sticks will
    forever haunt them.

    • ii

      ? nice name though

  • David Herrington

    PSVR is the budget high-end VR. It is still high-end… but you get what you pay for. Which is a lot less monetary investment but also a lot less bonuses. This is one of those bonuses that PC VR can do quite well but a PS4 will struggle with.

    That being said. PSVR is the best selling VR system as of this writing and whatever design decisions that Sony made I can guarantee you they do not regret. They will be laughing all the way to the bank.

    The “horrible sticks” that they are still using are old and need upgrading, but they are based on a design done 8 years ago and they didn’t have to do any more R&D on new ones which saved them money which is just another reason they can sell PSVR for so cheap.

    • I didn’t think the Move sticks were all that good, until I played Skyrim VR with them. It’s proof they can be integrated well with some thought. Just the simple idea of pointing your controller in a direction and moving forward in that direction, has made the game very playable. Also that big rubber ball on the end has saved some of my valuables. I have dents in my walls from the VIVE controllers.

  • Justin FlowerPristine

    I agree , they really need to revamp resolution, head tracking, it works but it could work way better, the way that the industry is moving so far forward with VR, eye tracking tech, the need look many years ahead in this department. to what everyone else is doing and be leader because they have a great hook in already.

    • David Herrington

      But that’s the thing. PSVR doesn’t have to be a tech leader… It just has to cost less and be almost as good as PC VR.

    • Ted Joseph

      Cheers. Lets hope Sony at the very least has an SKU that allows customers like us to purchase a better system.

  • Wow, this sounds really awesome! I hope it becomes public soon!

  • Ted Joseph

    Sold my PS4 Pro and PSVR and stayed with the Rift as the immersion is much better. However, I am excited to see what Sony brings out for the “PS5”. Hoping they have inside out tracking with cameras looking backwards for better tracking, wireless headset, 210 FOV, and built in earphones. If they go down this path I will jump back on their bandwagon…

  • Sion12

    off topic. but which one would be better to play video? S8+ with daydreams or PSVR(does gear vr lens make a different?)

    not sure which one is higher resolution

  • Hold on… what’s this about clustered PS4s? That sounds snazzy as hell!!! Where do I get the package+installer for that? (or SSH access + root to each PS4) I’d love to do some clustering… I’ve got the network for it. Gbit’s really kinda common now, and a lot of advanced gamers w/ a bit of IT experience would have no problem networking groups of 2 or 4 og PS4s into a VR render-cluster. The output might exceed a PS4Pro, and would enable NUMA storage, Proc & RAM resource pooling, essentially using a 5 port Gbit switch as a backbone. So long as the options were there, I don’t see an issue in making it all work. its simply a matter of an interface for a few existing unix packages used for distributed clustering to integrate it. Of course, it might ought to be handled as a code-based unlock on the core system os to enable it as an advanced user function to keep some poor joe from breaking their system screwing around with things they don’t understand… or something like that.