Immersive & Interactive Audio for Extended Reality

portal XR immersive projection audio

Share This Post

Share on facebook
Share on linkedin
Share on twitter
Share on email

Extended Reality (XR) is an umbrella term that embraces the concepts of virtual/augmented/mixed reality. Today XR technologies are being increasingly adopted in various fields, such as entertainment, health, social network, education/training, marketing and tourism. The common aim of XR applications is to provide the technology user with an immersive and interactive experience, i.e., sense of being in a virtual or augmented environment, interacting with virtual beings or objects. To allow for a high level of multi-sensory immersive experience, it is crucial for XR technologies to produce auditory cues necessary for accurate sound localisation and the perception of various spatial attributes, as well as realistic visual and haptic cues. Furthermore, sensorimotor contingencies afforded by XR technologies (e.g. head and motion tracking-based controls) allow for a realistic interaction with the virtual environment and beings, which would lead to a more immersive experience.

The key is to explore the recent developments in immersive and interactive audio technologies for XR applications, and to provide new insights into future research to advance this field. Although the last decade saw a great advance in three-dimensional (3D) audio technologies for multichannel and binaural recording, reproduction and transmission, there still exist various challenges specific to XR contexts that need to be overcome through further research, e.g. natural and efficient rendering of dynamic cues for 6-degrees-of-freedom (6DoF) experience, individualisation and optimisation of head-related transfer functions, sound field capture and post-processing techniques, efficient virtual acoustic modeling, etc. Furthermore, rapid advances in artificial intelligence, human-computer interaction, internet of things, and cloud and edge processing technologies open up new opportunities for the development of new immersive and interactive audio technologies. In all of these, trade-off between perceived accuracy/quality and computational efficiency need to be considered, and therefore more research into perceptual-model-based approaches might be required.

Audio & acoustic sinal processing equipment for XR applications:
• Ambisonics
• Auditory-visual interaction
• Audio-haptic feedback
• Binaural synthesis techniques
• HRTF individualisation and optimisation
• Holographic auditory display
• Immersive audio using Internet-of-Things
• Machine learning-based approaches
• Microphone array techniques
• Objective metrics for auditory immersiveness measurements
• Six-degrees-of-freedom audio capture and processing
• Sound field synthesis methods
• Virtual acoustic simulation techniques

Audio is fully built into immersive Portal environments, but what is interesting in that taking a more targetted approach focused on the sensory system we can really understand the depth of impact that such an approach can have, and whilst most people think of audio as just noise, the variations in these technologies, peripheral virtual technologies and multi-frequency devices enables you to create a well-rounded sensory experience.

This is particularly important when it comes to devising applications, tools, simulations & training programs because audio is a far more powerful tool than what we are used to understanding- so when planning your immersive projection as well as other virtual XR technologies, remember that the ultimate goal is sensory immersion, it’s not about a novelty, it’s about being able to naturally interact.

Keywords: 3D Audio Technologies, Immersive Audio, Interactive Audio, Mixed Reality, Virtual Reality, Augmented Reality, Virtual Acoustics

This research topic is hosted on www.frontiersin.org where further information, reading and subject materials can be found.

Subscribe To Our Newsletter

Get updates and learn from the best

More To Explore

LET'S COLLAB

Join The Co.llaborator Program Today!

The Program is all set to develop the product with your collab in a manner
which creates a win-win situation for both the parties.