While graphic fidelity goes a long way toward creating realistic, immersive, and compelling experiences in VR, sound also plays an important role. Oculus recently introduced some significant breakthroughs in spatialized audio to the Rift SDK, including Near-Field HRTF and Volumetric Sound Sources.
Sound is crucial when it comes to creating a believable VR experience. Spatialized audio replicates how sound waves interact with the environment as well as your head and ears, so that you really feel like you’re in the virtual world.
While video games and conventional film present audio on a plane, VR uses head-related transfer function (HRTF) to simulate how sound would reach the ears, which allows for a full 3D audio experience. The Rift headset is designed so that the sound you hear comes from the same direction as a given visual stimulus. Thanks to built in headphones, that means designers can place sounds behind you as a prompt to turn toward something beyond your field of view.
All of this gives developers the tools they need to make their experiences more realistic, expressive, and immersive. And that’s just the beginning.
“When audio’s done right in flat media, you don’t even notice it,” explains Stirling. “VR is a multiplying factor—when you’re actually in the space, your mind naturally accepts sounds as natural.”
Somewhat surprisingly, the believability of spatialized audio can even work against your overall sound design if you neglect to take the corresponding visuals of a scene into account. Stirling points to Oculus Dreamdeck as an early example—while one particular scene was meant to show off spatialized sound as an object moves around you, the effect was largely lost of audiences who followed the animated object with their head.
For developers entering the VR space, Smurdon recommends involving audio much sooner in the development cycle than might be the norm with more conventional forms of media. “Slapping it on at the end was never a good idea, but it’s even worse in VR,” he explains. “Since you have a 360° view, audio is really helpful in directing the player to look in the right direction. Audio plays a bigger role than ever given the absence of traditional directorial cues.”