Video games already use synthetic six degrees of freedom audio to immerse players, but the tech from Zylia and Infineon allows the use of live recorded navigable 6DoF sound from an actual physical space.
When entering virtual worlds, you need to do so not just with your eyes, but also with your ears. If sound positioning doesn’t work in sync with the image, the experience will not be completely “immersive”. In fact, most VR cinematic audio-video or live recordings content available presents, says Zylia, a 3DoF (3 Degrees of Freedom) scenario, in which the user occupies a single, fixed point of view allowing rotation, but not translation movements.
https://youtu.be/Zvk0OL_mubc
That’s where six degrees of freedom technology, or 6DoF technology for short, enters. Used in Virtual and Augmented Reality, it enables the user to experience 3D audio/video content in the virtual space, simply by introducing an unlimited number of listening and vantage points.
Those who have tried virtual spaces are well aware of the limitations of early examples, and how the technology has evolved so the high-level image that helps to full immerse the viewers is followed by a realistic sound. We’re still moving forward in terms of image quality, with 8K head mounted displays, for example, arriving this year, but, according to Zylia, “until now, image-related technologies have been developing much faster, leaving the sound far behind.”
Creating 6DoF recordings
The company has made the first attempt to show that 6DoF with an immersive sound experience can be achieved, and will present its innovative and unique approach to Six Degrees of Freedom (6DoF) technology in virtual and augmented reality (VR/AR) during the Mobile World Congress 2019, which will run February 25-28 at Fira Gran Via and Fira Montjuïc conference centers in Barcelona, Spain.
https://youtu.be/4RgMqrSydnQ
For this project Zylia, an industry leader in the field of 3D audio recording and production, worked together with Infineon Technologies AG, a world leader in semiconductor solutions. In order to prepare its 6DoF demonstration, Zylia used multiple simultaneous and synchronized Higher Order Ambisonics recordings made with nine ZYLIA ZM-1 microphones evenly spaced in the room. Zylia’s work resulted in creating a Unity-based audiovisual system, enabling users to navigate within a simulated 3D space and listen to the 6DoF recordings from different perspectives.
From video games to immersive video content
“Synthetic 6 Degrees of Freedom audio, combining separate sound recordings into a single simulated sonic space, is very common in video games. What we will present at Mobile World Congress is a novel approach which enables not synthetic, but live recorded navigable 6DoF sound from an actual physical space,” explains Tomasz Zernicki, co-founder and chief technology officer of Zylia.
https://youtu.be/UGhw_iEL2rE
For the purpose of audio recording for 6DoF demonstration, Zylia provided a new product — 6DoF Development Kit. This is an important tool for people experimenting with spatial sound solutions, which could unite audio and video for immersive content.
A single ZYLIA ZM-1 is a 19-capsule microphone array with high-end 24-bit recording resolution, which allows users to record entire sound scenes with just one microphone. Equipped with Infineon’s class-leading 69-decibel SNR digital XENSIV MEMS microphones, the microphone array delivers high-fidelity and far-field audio recording. In addition, the array provides multiple microphone noise- and distortion-free audio signals for advanced audio signal processing, says the company.