Spatial Audio Explained: Top 5 VR Spatial Audio Platforms and Software
A massive percentage of our attention can be directed with audio cues, but a fully immersive experience requires spatial audio and not just cues. Ambisonic audio makes what a content consumer hears a believable auditory experience that matches what they see and experience. It allows a creator to realistically present audio cues within one’s content from any direction, drawing attention and providing a more immersive VR experience. Combined with HDR visuals, this medium becomes an extremely powerful engagement creator at a systemic level within the human brain.
What is spatial audio?
Spatial audio ( also referred to as 3D audio or 360 audio) is a sonic experience where the audio changes with the movement of the viewer’s head; 3D audio effects manipulate the audio waves produced by stereo speakers, surround-sound speakers, speaker-arrays, or headphones. This involves the simulated placement of virtual sound sources anywhere in a simulated three-dimensional space, including behind, above or below the listener. To account for direction, distance, and environmental factors, content creators produce such “spatially oriented” audio experiences when creating soundtracks.
Spatial music is music composed to intentionally exploit sound localization. This has been in use since prehistoric times in the form of antiphon. As a technique used in modern music, it has been in use since around 1928 as ‘Raumusik’ or “space music” from Germany.
The term spatialization is what is called the projection and localization of sound source in a space, physical or simulated and its spatial movement in space.
Combined, you end up with ambisonic audio techniques which are bracketed as having 5 levels+, the first level being the simulation of the points of a compass and increasing in steps to 8 then 16, 32, 64 +. This is technically known as spatial domain convolution of sound waves using head-related transfer functions (HRTF). It is the phenomenon of transforming sound waves utilizing HRTF with filters and crosstalk cancellation techniques, to mimic natural sounds waves through space, which emanate from a point in a virtual 3-D construct. It allows trickery of the brain using the ears and auditory nerves, seeming to place sounds in different 3-D locations upon hearing the sounds, even though the sounds may just be produced from just 2 speakers.
An immersive sphere of audio that can be delivered over headphones, just like humans hear sound in real life. In essence, it is possible to tell where a particular sound is coming from a point in space over a standard pair of headphones, without needing any external hardware or specialized piece of equipment. With 360 video and Virtual Reality, ambisonic audio also changes instantly in real-time, as the user looks around either wearing a VR headset or rotates a smartphone where they are viewing the 360 content.
Content platforms that support spatial audio
YouTube
YouTube itself creates first-order ambisonic audio, which encodes four spatially located sound sources separated at 90º angles to each other. Think of it as being able to push sounds from a chosen point of the compass. Higher-order ambisonics, which may be supported in the future, encode 8, 16, or more audio sources.
The Facebook 360 Spatial Workstation is a set of tools integrated into a software suite and it enables one to design spatial audio for 360 video and cinematic VR to a professional standard. It includes plugins support for industry standard audio workstations, a time synchronized 360 video player and utilities to design and publish ambisonic audio in all popular formats.
Audio crafted with these tools can be consumed on Facebook News Feed on Android and iOS devices, Chrome for desktop and the Samsung Gear VR headset through headphones. Its tools allow one to position soundtracks with what you see within a spherical format video. The 360 video player is synchronized to the DAW, meaning one is able to instantly preview mixes as you go. Imagine a dialogue track, recorded with a lapel mic, for a character who is walking around the scene. These tools allow you to place the sound, in time, with the movement of the character. In playback, the 360 Audio Engine takes care of head-tracking in real-time on the consumers’ device. The final audio is rendered with low latency across platforms, keeping the performance and quality platform agnostic.
The Google VR Audio System offers a sophisticated yet easy-to-use room simulation component to enhance the VR experience. It deals with the creation of rooms and then these rooms may be embedded within your spherical imagery.
Samsung VR supports mono, stereo, and multiple formats of spacial audio for playback. For audio content upload specs, check out Samsung VR’s content specs page.
VeeR also supports spatial audio and a simple search for ‘spatial audio’ yields some great examples such as:
And something more intense:
Spatial audio software and plug-ins
Adobe Premiere Pro
Here’s a video tutorial by Jason Levine on how to edit ambisonic audio for your 360/VR videos using Premiere Pro.
G’Audio Works is a powerful spatial audio plugin for digital audio workstations. The software is Mac-only that requires OS 10.10 and above. It leverages object-based mixing for detailed placement of sound in a 3D environment.
For those looking for a spatial audio solution for game engines, check out G’Audio Craft, which is specially designed for gaming sonic experiences.
G’Audio’s official YouTube channel released a series of tutorials to walk you through using its spatial audio plug-ins. The one below is the first of the series.
However, they do offer G-Audio Encoder that’s compatible with both Mac and Windows.
Steam Audio
Steam Audio allegedly delivers best-in-class spatialized audio experiences through a different branch of ambisonics, enabling a more fully-featured spatial audio system attuned to the physical attributes of your virtualized geometry. The steam audio package can combine multiple occlusion, reflection, reverb and HRTF effects for a natural sounding immersion.
Pro Tools now include support for ambisonic and VR/360 Audio. For detailed workflow and tutorials, visit Avid’s official website.
The Ambisonic Toolkit (ATK) is a free spatial audio plug-in for Reaper and Super Collider, which brings together a number of classic and novel tools for the artist working with Ambisonic surround sound. It also works with the GoPro VR Player. The toolset is intended to be both ergonomic and comprehensive, framed so that the user is enabled to ‘think Ambisonic’.
For addtional resources, Soundfly has a great piece covering how to mix 360 audio for spatial audio, which also analyzes several popular spatial audio plugins; also this Medium article on how spatial audio works in VR is an interesting read.
Any questions and thoughts, drop them in the comments below!
Adobe, Facebook, G'Audio, Spatial Audio, Steam, VeeR, YouTube
Actually I do have a comment, I’m trying to use ambisonics and spatial audio in Logic X. Im not really having much luck using the aforementioned plug-ins but maybe apple like every other apple department have forgotten about VR or emersion in many ways and this is just simply forgotten about in logic X. Obviously we have binaural and 5.1/7.1 surround but I’m looking for something far more intuitive to build music on logic in a VR environment.
Any ideas?
Thanks!
Two things that I would like to add to the article, and maybe you can also check them out as they are the standard in immersive audio authoring, are the Reaper DAW and the excellent Blue Ripple Sound plugins. Very fair prices, awesome support and extremely easy to use and versatile workflow.