Perception markup language: towards a standardized representation of perceived nonverbal behaviors

Authors:
Stefan Scherer;Stacy Marsella;Giota Stratou;Yuyu Xu;Fabrizio Morbini;Alesia Egan;Albert (Skip) Rizzo;Louis-Philippe Morency
Affiliations:
Institute for Creative Technologies, University of Southern California, Los Angeles, California;Institute for Creative Technologies, University of Southern California, Los Angeles, California;Institute for Creative Technologies, University of Southern California, Los Angeles, California;Institute for Creative Technologies, University of Southern California, Los Angeles, California;Institute for Creative Technologies, University of Southern California, Los Angeles, California;Institute for Creative Technologies, University of Southern California, Los Angeles, California;Institute for Creative Technologies, University of Southern California, Los Angeles, California;Institute for Creative Technologies, University of Southern California, Los Angeles, California
Venue:
IVA'12 Proceedings of the 12th international conference on Intelligent Virtual Agents
Year:
2012

Citing 8
Cited 1

SmartBody: behavior realization for embodied conversational agents

Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 1
The Next Step towards a Function Markup Language

IVA '08 Proceedings of the 8th international conference on Intelligent Virtual Agents
Honest Signals: How They Shape Our World

Honest Signals: How They Shape Our World
Modeling wisdom of crowds using latent mixture of discriminative experts

HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers - Volume 2
Towards a common framework for multimodal generation: the behavior markup language

IVA'06 Proceedings of the 6th international conference on Intelligent Virtual Agents
Nonverbal behavior generator for embodied conversational agents

IVA'06 Proceedings of the 6th international conference on Intelligent Virtual Agents
Spotting laughter in natural multiparty conversations: A comparison of automatic online and offline approaches using audiovisual data

ACM Transactions on Interactive Intelligent Systems (TiiS) - Special Issue on Affective Interaction in Natural Environments
FAAST: The Flexible Action and Articulated Skeleton Toolkit

VR '11 Proceedings of the 2011 IEEE Virtual Reality Conference

Understanding the nonverbal behavior of socially anxious people during intimate self-disclosure

IVA'12 Proceedings of the 12th international conference on Intelligent Virtual Agents

Quantified Score

Hi-index	0.00

Visualization

Abstract

Modern virtual agents require knowledge about their environment, the interaction itself, and their interlocutors' behavior in order to be able to show appropriate nonverbal behavior as well as to adapt dialog policies accordingly. Recent achievements in the area of automatic behavior recognition and understanding can provide information about the interactants' multimodal nonverbal behavior and subsequently their affective states. In this paper, we introduce a perception markup language (PML) which is a first step towards a standardized representation of perceived nonverbal behaviors. PML follows several design concepts, namely compatibility and synergy, modeling uncertainty, multiple interpretative layers, and extensibility, in order to maximize its usefulness for the research community. We show how we can successfully integrate PML in a fully automated virtual agent system for healthcare applications.