Multimodal Human Machine Interactions in Virtual and Augmented Reality

  • Authors:
  • Gérard Chollet;Anna Esposito;Annie Gentes;Patrick Horain;Walid Karam;Zhenbo Li;Catherine Pelachaud;Patrick Perrot;Dijana Petrovska-Delacrétaz;Dianle Zhou;Leila Zouari

  • Affiliations:
  • CNRS-LTCI TELECOM-ParisTech, Paris, France 75634;Dept. of Psycology, and IIASS, Second University of Naples, Italy;CNRS-LTCI TELECOM-ParisTech, Paris, France 75634;TELECOM & Management SudParis, Evry, France;CNRS-LTCI TELECOM-ParisTech, Paris, France 75634;TELECOM & Management SudParis, Evry, France;CNRS-LTCI TELECOM-ParisTech, Paris, France 75634 and LINC, IUT de Montreuil, Université de Paris 8, Montreuil, France 93100;CNRS-LTCI TELECOM-ParisTech, Paris, France 75634 and Institut de Recherche Criminelle de la Gendarmerie Nationale (IRCGN), Rosny sous Bois, France;TELECOM & Management SudParis, Evry, France;TELECOM & Management SudParis, Evry, France;CNRS-LTCI TELECOM-ParisTech, Paris, France 75634

  • Venue:
  • Multimodal Signals: Cognitive and Algorithmic Issues
  • Year:
  • 2009

Quantified Score

Hi-index 0.01

Visualization

Abstract

Virtual worlds are developing rapidly over the Internet. They are visited by avatars and staffed with Embodied Conversational Agents (ECAs). An avatar is a representation of a physical person. Each person controls one or several avatars and usually receives feedback from the virtual world on an audio-visual display. Ideally, all senses should be used to feel fully embedded in a virtual world. Sound, vision and sometimes touch are the available modalities. This paper reviews the technological developments which enable audio-visual interactions in virtual and augmented reality worlds. Emphasis is placed on speech and gesture interfaces, including talking face analysis and synthesis.