Towards Natural Gesture Synthesis: Evaluating Gesture Units in a Data-Driven Approach to Gesture Synthesis

Authors:
Michael Kipp;Michael Neff;Kerstin H. Kipp;Irene Albrecht
Affiliations:
DFKI, Germany;UC Davis, USA;Saarland University, Experimental Neuropsychology Unit, Germany;TomTec Imaging Systems GmbH, Germany
Venue:
IVA '07 Proceedings of the 7th international conference on Intelligent Virtual Agents
Year:
2007

Citing 18
Cited 26

Numerical recipes in C (2nd ed.): the art of scientific computing

Numerical recipes in C (2nd ed.): the art of scientific computing
Animated conversation: rule-based generation of facial expression, gesture & spoken intonation for multiple conversational agents

SIGGRAPH '94 Proceedings of the 21st annual conference on Computer graphics and interactive techniques
Animating human athletics

SIGGRAPH '95 Proceedings of the 22nd annual conference on Computer graphics and interactive techniques
The EMOTE model for effort and shape

Proceedings of the 27th annual conference on Computer graphics and interactive techniques
Truth is beauty: researching embodied conversational agents

Embodied conversational agents
BEAT: the Behavior Expression Animation Toolkit

Proceedings of the 28th annual conference on Computer graphics and interactive techniques
Modeling tension and relaxation for computer animation

Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation
Motion capture-driven simulations that hit and react

Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation
Creating Interactive Virtual Humans: Some Assembly Required

IEEE Intelligent Systems
Movement Phase in Signs and Co-Speech Gestures, and Their Transcriptions by Human Coders

Proceedings of the International Gesture Workshop on Gesture and Sign Language in Human-Computer Interaction
Speaking with hands: creating animated conversational characters from recordings of human performance

ACM SIGGRAPH 2004 Papers
Synthesizing multimodal utterances for conversational agents: Research Articles

Computer Animation and Virtual Worlds
AER: aesthetic exploration and refinement for expressive character animation

Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation
Design and evaluation of expressive gesture synthesis for embodied conversational agents

Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems
From brows to trust: evaluating embodied conversational agents

From brows to trust: evaluating embodied conversational agents
Gesture modeling and animation based on a probabilistic re-creation of speaker style

ACM Transactions on Graphics (TOG)
Nonverbal behavior generator for embodied conversational agents

IVA'06 Proceedings of the 6th international conference on Intelligent Virtual Agents
Implementing expressive gesture synthesis for embodied conversational agents

GW'05 Proceedings of the 6th international conference on Gesture in Human-Computer Interaction and Simulation

Gesture modeling and animation based on a probabilistic re-creation of speaker style

ACM Transactions on Graphics (TOG)
Predicting Listener Backchannels: A Probabilistic Multimodal Approach

IVA '08 Proceedings of the 8th international conference on Intelligent Virtual Agents
IGaze: Studying Reactive Gaze Behavior in Semi-immersive Human-Avatar Interactions

IVA '08 Proceedings of the 8th international conference on Intelligent Virtual Agents
Evaluating Data-Driven Style Transformation for Gesturing Embodied Agents

IVA '08 Proceedings of the 8th international conference on Intelligent Virtual Agents
IDEAS4Games: Building Expressive Virtual Characters for Computer Games

IVA '08 Proceedings of the 8th international conference on Intelligent Virtual Agents
Multimodal Human Machine Interactions in Virtual and Augmented Reality

Multimodal Signals: Cognitive and Algorithmic Issues
Some Pitfalls for Developing Enculturated Conversational Agents

Proceedings of the 13th International Conference on Human-Computer Interaction. Part III: Ubiquitous and Intelligent Interaction
Multimodal Corpus Analysis as a Method for Ensuring Cultural Usability of Embodied Conversational Agents

HCD 09 Proceedings of the 1st International Conference on Human Centered Design: Held as Part of HCI International 2009
EMBR --- A Realtime Animation Engine for Interactive Embodied Agents

IVA '09 Proceedings of the 9th International Conference on Intelligent Virtual Agents
Augmenting Gesture Animation with Motion Capture Data to Provide Full-Body Engagement

IVA '09 Proceedings of the 9th International Conference on Intelligent Virtual Agents
A Combined Semantic and Motion Capture Database for Real-Time Sign Language Synthesis

IVA '09 Proceedings of the 9th International Conference on Intelligent Virtual Agents
Evaluating models of speaker head nods for virtual agents

Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1
Learning backchannel prediction model from parasocial consensus sampling: a subjective evaluation

IVA'10 Proceedings of the 10th international conference on Intelligent virtual agents
Evaluating the effect of gesture and language on personality perception in conversational agents

IVA'10 Proceedings of the 10th international conference on Intelligent virtual agents
Why is the creation of a virtual signer challenging computer animation?

MIG'10 Proceedings of the Third international conference on Motion in games
The SignCom system for data-driven animation of interactive virtual signers: Methodology and Evaluation

ACM Transactions on Interactive Intelligent Systems (TiiS)
Don't scratch! self-adaptors reflect emotional stability

IVA'11 Proceedings of the 10th international conference on Intelligent virtual agents
Modeling parallel state charts for multithreaded multimodal dialogues

ICMI '11 Proceedings of the 13th international conference on multimodal interfaces
Requirements for a gesture specification language: a comparison of two representation formalisms

GW'09 Proceedings of the 8th international conference on Gesture in Embodied Communication and Human-Computer Interaction
Performatology: an arts approach to designing performative embodied agents (PEAs) for procedural character animation

Proceedings of the 6th International Conference on Foundations of Digital Games
Sleight of hand: perception of finger motion from reduced marker sets

I3D '12 Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games
Performatology: a procedural acting approach for interactive drama in cinematic games

ICIDS'11 Proceedings of the 4th international conference on Interactive Digital Storytelling
Modeling speaker behavior: a comparison of two approaches

IVA'12 Proceedings of the 12th international conference on Intelligent Virtual Agents
Virtual character performance from speech

Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation
Cultural diversity for virtual characters (extended abstract)

IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Gesture synthesis adapted to speech emphasis

Speech Communication

Quantified Score

Hi-index	0.00

Visualization

Abstract

Virtual humans still lack naturalness in their nonverbal behaviour. We present a data-driven solution that moves towards a more natural synthesis of hand and arm gestures by recreating gestural behaviour in the style of a human performer. Our algorithm exploits the concept of gesture units to make the produced gestures a continuous flow of movement. We empirically validated the use of gesture units in the generation and show that it causes the virtual human to be perceived as more natural.