Efficient and practical audio-visual rendering for games using crossmodal perception

  • Authors:
  • David Grelaud;Nicolas Bonneel;Michael Wimmer;Manuel Asselot;George Drettakis

  • Affiliations:
  • REVES/INRIA Sophia-Antipolis;REVES/INRIA Sophia-Antipolis;VUT;REVES/INRIA Sophia-Antipolis;REVES/INRIA Sophia-Antipolis

  • Venue:
  • Proceedings of the 2009 symposium on Interactive 3D graphics and games
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

Interactive applications such as computer games, are inherently audio visual, requiring high-quality rendering of complex 3D audio soundscapes and graphics environments. A frequent source of audio events is impact sounds, typically generated with physics engines. In this paper, we first present an optimization allowing efficient usage of impact sounds in a unified audio rendering pipeline, also including prerecorded sounds. We also exploit a recent result on audio-visual crossmodal perception to introduce a new level-of-detail selection algorithm, which jointly chooses the quality level of audio and graphics rendering. We have integrated these two techniques as a comprehensive crossmodal audio-visual rendering pipeline in a home-grown game engine, thus demonstrating the potential utility of our approach.