SIGGRAPH '94 Proceedings of the 21st annual conference on Computer graphics and interactive techniques
Improv: a system for scripting interactive actors in virtual worlds
SIGGRAPH '96 Proceedings of the 23rd annual conference on Computer graphics and interactive techniques
Video Rewrite: driving visual speech with audio
Proceedings of the 24th annual conference on Computer graphics and interactive techniques
Proceedings of the 26th annual conference on Computer graphics and interactive techniques
BEAT: the Behavior Expression Animation Toolkit
Proceedings of the 28th annual conference on Computer graphics and interactive techniques
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Formational Parameters and Adaptive Prototype Instantiation for MPEG-4 Compliant Gesture Synthesis
CA '02 Proceedings of the Computer Animation
Acquiring and validating motion qualities from live limb gestures
Graphical Models
Mood swings: expressive speech animation
ACM Transactions on Graphics (TOG)
Synthesizing multimodal utterances for conversational agents: Research Articles
Computer Animation and Virtual Worlds
Hidden Conditional Random Fields for Gesture Recognition
CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2
Presence: Teleoperators and Virtual Environments
Responsive characters from motion fragments
ACM SIGGRAPH 2007 papers
Near-optimal character animation with continuous control
ACM SIGGRAPH 2007 papers
Gesture modeling and animation based on a probabilistic re-creation of speaker style
ACM Transactions on Graphics (TOG)
Data-Driven 3D Facial Animation
Data-Driven 3D Facial Animation
Analysis of Head Gesture and Prosody Patterns for Prosody-Driven Head-Gesture Animation
IEEE Transactions on Pattern Analysis and Machine Intelligence
Dynamic Programming and Optimal Control, Vol. II
Dynamic Programming and Optimal Control, Vol. II
Real-time prosody-driven synthesis of body language
ACM SIGGRAPH Asia 2009 papers
Implementing expressive gesture synthesis for embodied conversational agents
GW'05 Proceedings of the 6th international conference on Gesture in Human-Computer Interaction and Simulation
Learning dynamic audio-visual mapping with input-output Hidden Markov models
IEEE Transactions on Multimedia
How to train your avatar: a data driven approach to gesture generation
IVA'11 Proceedings of the 10th international conference on Intelligent virtual agents
A framework for motion based bodily enaction with virtual characters
IVA'11 Proceedings of the 10th international conference on Intelligent virtual agents
Prosody-driven robot ARM gestures generation in human-robot interaction
HRI '12 Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction
Data-driven finger motion synthesis for gesturing characters
ACM Transactions on Graphics (TOG) - Proceedings of ACM SIGGRAPH Asia 2012
Virtual character performance from speech
Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation
Towards higher quality character performance in previz
Proceedings of the Symposium on Digital Production
Customizing by doing for responsive video game characters
International Journal of Human-Computer Studies
Gesture synthesis adapted to speech emphasis
Speech Communication
Hi-index | 0.00 |
We introduce gesture controllers, a method for animating the body language of avatars engaged in live spoken conversation. A gesture controller is an optimal-policy controller that schedules gesture animations in real time based on acoustic features in the user's speech. The controller consists of an inference layer, which infers a distribution over a set of hidden states from the speech signal, and a control layer, which selects the optimal motion based on the inferred state distribution. The inference layer, consisting of a specialized conditional random field, learns the hidden structure in body language style and associates it with acoustic features in speech. The control layer uses reinforcement learning to construct an optimal policy for selecting motion clips from a distribution over the learned hidden states. The modularity of the proposed method allows customization of a character's gesture repertoire, animation of non-human characters, and the use of additional inputs such as speech recognition or direct user control.