noctvrnal logo

Noctvrnal: audio in AR/VR is 51% of the experience

I had the amazing pleasure of interviewing Helena McGill and Anna Wozniewicz of Noctvrnal, a startup focused in audio and music production, with special attention to immersive experiences.

noctvrnal helena anna
Helena and Anna from Noctvrnal (Image by Noctvrnal)

When I develop a VR experience, I usually give a lot of attention to the visual aspect, while I often leave the sound and music always as the last thing to care about. After having talked with Helena and Anna, I realized that I’ve always made a huge error, since spatial high-quality audio is fundamental to improve the retention and the appreciation of the users.

In this interview, you will discover why audio is so important and you will get also various insights on audio in immersive experiences and on the problems of being a woman working in XR. Have a nice reading!

What does your company Noctvrnal do? What are you currently working on?

Noctvrnal is an audio post-production studio based in Los Angeles. Co-founders Helena McGill and Anna Wozniewicz are visual storytellers with a background in sound design for narrative film. After noticing a lack of female entrepreneurs in audio and music production, the duo decided to launch Noctvrnal in January 2017 and have since expanded into the XR and immersive tech space. Noctvrnal is bringing immersive sound design to the forefront of virtual and augmented reality by pushing the boundaries of current techniques to create a more visceral experience. The team is passionate about creating new worlds through sound, and their work has been featured at MIT, USC, and VRLA.

Currently, we are working on a number of VR experiences, a permanent public works installation in Denver, some experimental immersive theater pieces, a series of audio pod-plays, and a sensor-activated audio-reactive installation.

You are an audio designer… why do you think that audio design in an AR/VR application is important?

Audio in AR/VR is 51% of the experience. As in real life, when you look around, your field of view is limited, but you’re still hearing the full 360 degrees sound field around you, so it needs to be realistic and not pull you out of the experience.

All you need to know on HoloLens 2
In an AR experience, we usually focus on visuals and interactions. Actually, audio is of paramount importance, both for consumer and enterprise applications.
Is it also important for enterprise applications or only for games and movies? Why?

Spatial audio is important for all AR/VR experiences. Research has shown that spatial audio keeps users in an experience for 40% longer. You want the sounds to mimic real life and add realism/immersion to the experience, whether it’s for entertainment or enterprise applications.

We pull a lot from our research background in psychoacoustics in order to design effective auditory UX/UI for enterprise. This is especially important in the military, training, and therapy realms, and we draw from our background in psychoacoustics in order to design these systems.

So, how the audio for an AR/VR training experience should be designed? What pieces of advice can you give to us?

For any AR/VR experience, you want to keep psychoacoustics in mind – placing sounds in a way that will draw viewers’ attention, or making sure the sound effect’s duration is enough to hold their attention and allow them to locate the source. For example, sound sources are more easily located along a horizontal axis, but our brains have a harder time perceiving variations in height. Use each experience as R&D for the next one – every experience is different, so figure out what worked (and why), what didn’t, and how you can improve for the next one.

And what are the differences of designing the audio for an AR experience from designing it for a VR one?

Depends on the experience itself. The main difference in workflow, whether AR or VR, is if the experience is interactive and being built in a game engine, or has a more linear/cinematic timeline. Also, within a game engine, you can simulate room reflections and occlusion more accurately for VR, because the objects are all virtual. For AR, most experiences exist as mobile AR and the audio is an afterthought, because people don’t always carry headphones around with them. We’re really excited about BoseAR and its potential to bring more attention to hands-free, spatial audio.

Are you excited about the HoloLens 2? Why?

Very excited! We work a lot with enterprise and training, so the Hololens 2 is going to open up a lot of opportunities and potential for advancement in that sector.

noctvrnal magic leap
Helena playing around with Magic Leap One. This device also features Tonandi, an experience all focused on audio (Image by Noctvrnal)
How do you perform your work?

We like to be involved as early as possible in the script development process. That way, we can include audio triggers as part of the story and use them to direct the viewer’s attention. We capture audio on set for cinematic VR, create sound objects for game engine-based experiences, and work with ambisonics (sound fields). Our primary tools are DAWs, spatial audio plugins and SDKs, as well as traditional and ambisonic microphones.

What in your opinion still lacks from current AR/VR technology to make the audio fully realistic?

Biggest issue: many experiences are still not using spatial audio; it’s just in stereo. We’ve also found that spatial audio isn’t fully understood by the creative teams, or isn’t utilized to its fullest potential, i.e. to engage the user with the full 360-degree space.

Spatial audio SDKs are rapidly improving in terms of realism. However, the next step will be headsets that measure HRTFs in real time.

What is your opinion on the recent audio AR glasses like the ones by Bose?

They’re a great start and a huge step forward for spatial audio applications. They’re making spatial audio more accessible to everyone, and putting an emphasis on the audio side of things, rather than focusing on visuals first.

What is in your opinion the best AR and VR headset for what concerns the audio quality?

It really depends on the headphones you’re using with that headset. Standalone headsets, like the Oculus Go, have decent speakers built in, but oftentimes for demo purposes at crowded events, the noise floor isn’t taken into account.

noctvrnal how to audio in xr
Audio visual installation created by Noctvrnal (Image by Noctvrnal)
There is a lot of buzz about diversity in augmented and virtual reality. You are a woman in XR. What do you want to say about the topic?

Being on the crossroads between three male-dominated industries (tech, entertainment, and audio) definitely isn’t easy.

We’ve had to expend a lot of energy on simply proving our worth before we’re taken seriously as audio engineers and treated as equals and collaborators. One of the biggest problems for women in XR, too, is access – most of the funding comes from men, so fewer women are getting funded than men, which creates a vicious cycle. We’ve found that our biggest projects and repeat clients come from fellow women in the XR space. Whether intentionally or not, women supporting women has been the biggest support system in immersive tech.

How do you envision the future for audio in XR?

The future of immersive tech and spatial audio lies in locationbased experiences and experiential marketing events. Headsets are isolating, so the more we can bring people together and use tech to create social experiences, the more engaging the experiences will be. Some of the best experiences currently are audio reactive, or concerts in VR, or music-oriented games liked Beat Saber (the #1 game on Steam).

What about your future instead?

We’d like to continue pushing the boundaries of spatial audio and finding new ways to incorporate audio into experiences, whether they’re AR/VR, location-based, or completely experimental content. We are storytellers at heart, so we’re always looking for ways to challenge current techniques and use sound to convey stories in new, engaging ways.


I hope that you have loved this interview and that it has helped you in getting more insights on audio in Augmented and Virtual Reality experiences. And if you want to get to know better the fantastic girls at Noctvrnal, have a look at their website or ask me for an introduction! 🙂


Disclaimer: this blog contains advertisement and affiliate links to sustain itself. If you click on an affiliate link, I'll be very happy because I'll earn a small commission on your purchase. You can find my boring full disclosure here.

Releated

playstation vr 2

The XR Week Peek (2024.03.26): Sony halts production of PSVR 2, Meta slashes the price of Quest 2, and more!

This has been the week of GDC and of the NVIDIA GTC, so we have a bit more pieces of news than the usual because there have been some interesting things announced there. Oh yes, also IEEE VR had its show. Actually, a lot of things happened for tech people in the US, which is […]

We need camera access to unleash the full potential of Mixed Reality

These days I’m carrying on some experiments with XR and other technologies. I had some wonderful ideas of Mixed Reality applications I would like to prototype, but most of them are impossible to do in this moment because of a decision that almost all VR/MR headset manufacturers have taken: preventing developers from accessing camera data. […]