From annotated multimodal corpora to simulated human-like behaviors

Authors:
Matthias Rehm;Elisabeth André
Affiliations:
Augsburg University, Institute of Computer Science, Augsburg, Germany;Augsburg University, Institute of Computer Science, Augsburg, Germany
Venue:
ZiF'06 Proceedings of the Embodied communication in humans and machines, 2nd ZiF research group international conference on Modeling communication with robots and virtual humans
Year:
2006

Citing 12
Cited 3

Improvising linguistic style: social and affective bases for agent personality

AGENTS '97 Proceedings of the first international conference on Autonomous agents
The EMOTE model for effort and shape

Proceedings of the 27th annual conference on Computer graphics and interactive techniques
BEAT: the Behavior Expression Animation Toolkit

Proceedings of the 28th annual conference on Computer graphics and interactive techniques
Eyes alive

Proceedings of the 29th annual conference on Computer graphics and interactive techniques
The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Speaking with hands: creating animated conversational characters from recordings of human performance

ACM SIGGRAPH 2004 Papers
Towards a model of face-to-face grounding

ACL '03 Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - Volume 1
Where do they look?: Gaze behaviors of multiple users interacting with an embodied conversational agent

Lecture Notes in Computer Science
Multimodal expressive embodied conversational agents

Proceedings of the 13th annual ACM international conference on Multimedia
Explorations in engagement for humans and robots

Artificial Intelligence
Perception of blended emotions: from video corpus to expressive agent

IVA'06 Proceedings of the 6th international conference on Intelligent Virtual Agents
Creativity meets automation: combining nonverbal action authoring with rules and machine learning

IVA'06 Proceedings of the 6th international conference on Intelligent Virtual Agents

Multimodal Corpus Analysis as a Method for Ensuring Cultural Usability of Embodied Conversational Agents

HCD 09 Proceedings of the 1st International Conference on Human Centered Design: Held as Part of HCI International 2009
Towards personality-based user adaptation: psychologically informed stylistic language generation

User Modeling and User-Adapted Interaction
Turk-2, a multi-modal chess player

International Journal of Human-Computer Studies

Quantified Score

Hi-index	0.00

Visualization

Abstract

Multimodal corpora prove useful at different stages of the development process of embodied conversational agents. Insights into human-human communicative behaviors can be drawn from such corpora. Rules for planning and generating such behavior in agents can be derived from this information. And even the evaluation of human-agent interactions can rely on corpus data from human-human communication. In this paper, we exemplify how corpora can be exploited at the different development steps, starting with the question of how corpora are annotated and on what level of granularity. The corpus data can be used either directly for imitating the human behavior recorded in the corpus or rules can be derived from the data which govern the behavior planning process. Corpora can even play a vital role in the evaluation of agent systems. Several studies are presented that make use of corpora for the evaluation task.