Multimodal sensing, interpretation and copying of movements by a virtual agent

Authors:
Elisabetta Bevacqua;Amaryllis Raouzaiou;Christopher Peters;George Caridakis;Kostas Karpouzis;Catherine Pelachaud;Maurizio Mancini
Affiliations:
LINC, IUT de Montreuil, Université de Paris 8;Image, Video and Multimedia Systems Laboratory, National Technical University of Athens, Greece;LINC, IUT de Montreuil, Université de Paris 8;Image, Video and Multimedia Systems Laboratory, National Technical University of Athens, Greece;Image, Video and Multimedia Systems Laboratory, National Technical University of Athens, Greece;LINC, IUT de Montreuil, Université de Paris 8;LINC, IUT de Montreuil, Université de Paris 8
Venue:
PIT'06 Proceedings of the 2006 international tutorial and research conference on Perception and Interactive Technologies
Year:
2006

Citing 13
Cited 5

An approach to natural gesture in virtual environments

ACM Transactions on Computer-Human Interaction (TOCHI) - Special issue on virtual reality software and technology
Classifying Facial Actions

IEEE Transactions on Pattern Analysis and Machine Intelligence
The EMOTE model for effort and shape

Proceedings of the 27th annual conference on Computer graphics and interactive techniques
Emotion and personality in a conversational agent

Embodied conversational agents
FacEMOTE: qualitative parametric modifiers for facial animations

Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation
Formational Parameters and Adaptive Prototype Instantiation for MPEG-4 Compliant Gesture Synthesis

CA '02 Proceedings of the Computer Animation
Recognizing Facial Expression: Machine Learning and Application to Spontaneous Behavior

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02
Design and evaluation of expressive gesture synthesis for embodied conversational agents

Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems
Contextual recognition of head gestures

ICMI '05 Proceedings of the 7th international conference on Multimodal interfaces
Direction of attention perception for conversation initiation in virtual environments

Lecture Notes in Computer Science
Levels of representation in the annotation of emotion for the specification of expressivity in ECAs

Lecture Notes in Computer Science
Parameterized facial expression synthesis based on MPEG-4

EURASIP Journal on Applied Signal Processing
Implementing expressive gesture synthesis for embodied conversational agents

GW'05 Proceedings of the 6th international conference on Gesture in Human-Computer Interaction and Simulation

Too close for comfort?: adapting to the user's cultural background

Proceedings of the international workshop on Human-centered multimedia
Dancing the night away: controlling a virtual karaoke dancer by multimodal expressive cues

Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 3
What Should a Generic Emotion Markup Language Be Able to Represent?

ACII '07 Proceedings of the 2nd international conference on Affective Computing and Intelligent Interaction
Wave like an Egyptian: accelerometer based gesture recognition for culture specific interactions

BCS-HCI '08 Proceedings of the 22nd British HCI Group Annual Conference on People and Computers: Culture, Creativity, Interaction - Volume 1
Copying behaviour of expressive motion

MIRAGE'07 Proceedings of the 3rd international conference on Computer vision/computer graphics collaboration techniques

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present a scenario whereby an agent senses, interprets and copies a range of facial and gesture expression from a person in the real-world. Input is obtained via a video camera and processed initially using computer vision techniques. It is then processed further in a framework for agent perception, planning and behaviour generation in order to perceive, interpret and copy a number of gestures and facial expressions corresponding to those made by the human. By perceive, we mean that the copied behaviour may not be an exact duplicate of the behaviour made by the human and sensed by the agent, but may rather be based on some level of interpretation of the behaviour. Thus, the copied behaviour may be altered and need not share all of the characteristics of the original made by the human.