Real-Time Geometric Acoustics
March 26, 2014, GPU Technology Conference, San Jose, CA—Zoufu Cheng from the University of Illinois at Urbana-Champagne talked about making the audio as good as the video in games. The ability of the GPUs to process rays can be applied to acoustics as well as to the images to provide a more immersive game experience.
Sound in games has been assembled from sound tracks, dialog, ambient noises, and generated sounds to match the action. The problem is that the sounds are presented in a constant environment that does not take into account the differences in space and materials. Most games can produce some 3-D sound effects that reduce volume at a distance and use parameters to change other sound characteristics.
The differences between parametric settings and full simulations is dramatic. The parametric sounds from game play do not interact with the game state or local geometries. In comparison, a simulation engine can take sources, game states, material properties, and geometry to change the sound characteristics that are much more realistic.
Parametric sounds are like sampling and have good positional audio to locate sound sources in free space. Problems occur when the parameters exceed the model limits, resulting in broken sound. The parameterized audio can only change amplitude and a small amount of time and filtering, as long as they are within the model. These sounds are much like a MP3, and are fixed.
Simulation can take into account game state information, highly chaotic sources can be decimated for better realism, due to the psycho-acoustic responses. The time difference between the ears and frequency filters that respond to position and material properties like hair or cloth versus stone create more realism. In addition, the simulation can differentiate between line of sight and occluded and reflected and refracted sounds. The reverb can be changed with volume and sounds can have echoes that change with the environment.
To simulate a normal environment requires about 3k triangles and 4k rays. Realistic simulations use about 3 orders of reflection and the heterogeneous response transfer function ( HRTF) causes about 12k total calculations. The resulting sounds change the sound of a voice with different environments and types of sounds in the ambient. The simulations use a form of backward geometry with audio ray tracing. The sims start from the receiver and not from the sources to reduce computational complexity.
This backwards ray tracing for the HRTF is physically inspired, but not fully physics correct. The processing traces line-of-sight paths to the sources and considers orders of reflections for each ray. The processing uses one ray per thread, and allows the placement of audio “cameras” to instrument positions. These functions can take into account the changes in sounds with position material properties. The computational complexity prevents the use of “perfect” reflections for the calculations.
The geometry helps to determine the computational requirements. If a ray misses an object, no further simulations occur for that ray. This structure allows for dynamically allocating rays to sources. In addition, the algorithms uses alternate rays from distant sounds and the amplitudes are changed as rays are assigned. One artifact is that a slower ray trace causes a warp divergence between bounding volumes, so they have to resort block at the source and need to use two ears for the simulations.
The processing chain uses a thread per ray and the path length is the length of a filter. Each ray generates an audio stream with delays and filters due to reflections that change with materials and HRTF. The filters are infinite impulse, and the impulse changes all locations per ray, but the rays and changes are not necessarily coherent. They changed from a finite to infinite filter with 532 bit floating point taps. The coefficients are generated in a custom tool.
The GPU mixes down the threads through atomic operations and integrate artificial reverberations. The simulations are ok to 3-4 orders which corresponds to a t60, the time to decrease 60 db, of about 500 ms for the reverb time. They apply these transforms to the source material, which differs fro the same reverb before HRTF.
Some design considerations for games. parametric sounds are easier for big battles that are chaotic. You only need to make the sounds more exciting or chaotic for best effect. Parametric sounds are only good for realistic and not cinematic sounds. For all other sounds and game types, a simulated sound field is much better.
