3-D sound for virtual reality and multimedia
3-D sound for virtual reality and multimedia
Spatial sound for video games and virtual environments utilizing real-time GPU-based convolution
Future Play '08 Proceedings of the 2008 Conference on Future Play: Research, Play, Share
| Hi-index | 0.00 |
Realistic 3D audio can greatly enhance the sense of presence in a virtual environment. We introduce a framework for capturing, transmitting and rendering of 3D audio in presence of other bandwidth savvy streams in a 3D Tele-immersion based virtual environment. This framework presents an efficient implementation for 3D Binaural Spatialization based on the positions of current objects in the scene, including animated avatars and on the fly reconstructed humans. We present a general overview of the framework, how audio is integrated in the system and how it can exploit the positions of the objects and room geometry to render realistic reverberations using head related transfer functions. The network streaming modules used to achieve lip-synchronization, high-quality audio frame reception, and accurate localization for binaural rendering are also presented. We highlight how large computational and networking challenges can be addressed efficiently. This represents a first step in adequate networking support for Binaural 3D Audio, useful for tele-presence. The subsystem is successfully integrated with a larger 3D immersive system, with state of art capturing and rendering modules for visual data.