Valve has announced support for AMD's TrueAudio Next technology in its Steam Audio SDK for developers. The partnership will allow game and VR application developers to reserve a portion of a GCN-based GPU's compute units for audio processing and increase the quality and quantity of audio sources as a result. AMD's OpenCL-based TrueAudio Next technology can run CPUs as well but it's strength is in the ability to run on a dedicated portion of the GPU to improve both frame times and audio quality since threads are not competing for the same GPU resources during complex scenes and the GPU can process complex audio scenes and convolutions much more efficiently than a CPU (especially as the number of sources and impulse responses increase) respectively.
Steam Audio's TrueAudio Next integration is being positioned as an option for developers and the answer to increasing the level of immersion in virtual reality games and applications. While TrueAudio Next is not using ray tracing for audio, it is physics-based and can be used to great effect to create realistic scenes with large numbers of direct and indirect audio sources, ambisonics, increased impulse response lengths, echoes, reflections, reverb, frequency equalization, and HRTF (Head Related Transfer Function) 3D audio. According to Valve indirect audio from multiple sources with convolution reverb is one of the most computationally intensive parts of Steam Audio, and TAN is able to handle it much more efficiently and accurately without affecting GPU frame times and freeing the CPU up for additional physics and AI tasks which it is much better at anyway. Convolution is a way of modeling and filtering audio to create effects such as echoes and reverb. In the case of indirect audio, Steam Audio uses ray tracing to generate an impulse response (it measures the distance and path audio would travel from source to listener) and then convolution is used to generate a reverb effect which, while very accurate, can be quite computationally intensive with it requiring hundreds of thousands of sound samples. Ambisonics further represent the directional nature of indirect sound which helps to improve positional audio and the immersion factor as sounds are more real-world modeled.
GPU versus CPU convolution (audio filtering) performance. Lower is better.
In addition to the ability of developers to dedicate a portion (up to 20 to 25%) of a GPU's compute units to audio processing, developers can enable/disable TrueAudio processing including the level of acoustic complexity and detail on a scene-by-scene basis. Currently it appears that Unity, FMOD Studio, and C API engines can hook into Steam Audio and the TrueAudio Next features, but it remains up to developers to use the features and integrate them into their games.
Note that GPU-based TrueAudio Next requires a GCN-based graphics card of the RX 470, RX 480, RX 570, RX 580, R9 Fury, R9 Fury X, Radeon Pro Duo, RX Vega 56, and RX Vega 64 variety in order to work, so that is a limiting factor in adoption much like the various hair and facial tech is for AMD and NVIDIA on the visual side of things where the question of is the target market large enough to encourage developers to put in the time and effort to enable X optional feature arises.
I do not pretend to be an audio engineer, nor do I play a GPU programmer on TV but more options are always good and I hope that developers take advantage of the resource reservation and GPU compute convolution algorithms of TrueAudio Next to further the immersion factor of audio as much as they have the visual side of things. As VR continues to become more relevant I think that developers will have to start putting more emphasis on accurate and detailed audio and that's a good thing for an aspect of gaming that has seemingly taken a backseat since Windows Vista.
What are your thoughts on the state of audio in gaming and Steam Audio's new TrueAudio Next integration?