Mechanisms for multimodality: taking fiction to another dimension
AFRIGRAPH '07 Proceedings of the 5th international conference on Computer graphics, virtual reality, visualisation and interaction in Africa
Towards human-like spoken dialogue systems
Speech Communication
| Hi-index | 0.00 |
This paper presents an architecture for interactive storytelling using state-of-the-art-technology in natural language processing, speech synthesis and 3D-character animation. A conversational 3D-character is used to tell nonlinear stories interactively. Our research focuses on multimodal conversation that combines verbal-vocal with nonverbal-nonvocal natural language. We present a framework for synchronizing speech with facial movements, gesture and body posture by combining findings from linguistics and psychology. The interaction process incorporates the parsing of a textual natural language utterance, the emotion-based triggering of a textual response and its vocal and visual performance - speech synthesis linked with lip-synchronous generation of facial expressions and gestures.