Guest Editorial: Gesture and speech in interaction: An overview

Authors:
Petra Wagner;Zofia Malisz;Stefan Kopp
Affiliations:
-;-;-
Venue:
Speech Communication
Year:
2014

Citing 44
Cited 0

Integrating simultaneous input from speech, gaze, and hand gestures

Intelligent multimedia interfaces
An approach to natural gesture in virtual environments

ACM Transactions on Computer-Human Interaction (TOCHI) - Special issue on virtual reality software and technology
Handbook of Multimodal and Spoken Dialogue Systems: Resources, Terminology and Product Evaluation

Handbook of Multimodal and Spoken Dialogue Systems: Resources, Terminology and Product Evaluation
Hand Tension as a Gesture Segmentation Cue

Proceedings of Gesture Workshop on Progress in Gestural Interaction
Movement Phase in Signs and Co-Speech Gestures, and Their Transcriptions by Human Coders

Proceedings of the International Gesture Workshop on Gesture and Sign Language in Human-Computer Interaction
Vision-Based Gesture Recognition: A Review

GW '99 Proceedings of the International Gesture Workshop on Gesture-Based Communication in Human-Computer Interaction
Real-Time Gesture Recognition by Means of Hybrid Recognizers

GW '01 Revised Papers from the International Gesture Workshop on Gesture and Sign Languages in Human-Computer Interaction
The Lexicon and the Alphabet of Gesture, Gaze, and Touch

IVA '01 Proceedings of the Third International Workshop on Intelligent Virtual Agents
Recovering the Temporal Structure of Natural Gesture

FG '96 Proceedings of the 2nd International Conference on Automatic Face and Gesture Recognition (FG '96)
3D Computer Graphics: A Mathematical Introduction with OpenGL

3D Computer Graphics: A Mathematical Introduction with OpenGL
Hidden Markov Model Based Continuous Online Gesture Recognition

ICPR '98 Proceedings of the 14th International Conference on Pattern Recognition-Volume 2 - Volume 2
Speaking with hands: creating animated conversational characters from recordings of human performance

ACM SIGGRAPH 2004 Papers
Towards integrated microplanning of language and iconic gesture for multimodal output

Proceedings of the 6th international conference on Multimodal interfaces
Contextual recognition of head gestures

ICMI '05 Proceedings of the 7th international conference on Multimodal interfaces
On-line Integration of Semantic Information from Speech and Gesture: Insights from Event-related Brain Potentials

Journal of Cognitive Neuroscience
Gesture modeling and animation based on a probabilistic re-creation of speaker style

ACM Transactions on Graphics (TOG)
Analysis of Head Gesture and Prosody Patterns for Prosody-Driven Head-Gesture Animation

IEEE Transactions on Pattern Analysis and Machine Intelligence
The Behavior Markup Language: Recent Developments and Challenges

IVA '07 Proceedings of the 7th international conference on Intelligent Virtual Agents
Estimating User's Conversational Engagement Based on Gaze Behaviors

IVA '08 Proceedings of the 8th international conference on Intelligent Virtual Agents
Increasing the expressiveness of virtual agents: autonomous generation of speech and gesture for spatial description tasks

Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 1
Presenting in style by virtual humans

COST 2102'07 Proceedings of the 2007 COST action 2102 international conference on Verbal and nonverbal communication behaviours
Analysis and synthesis of multimodal verbal and non-verbal interaction for animated interface agents

COST 2102'07 Proceedings of the 2007 COST action 2102 international conference on Verbal and nonverbal communication behaviours
Modeling embodied feedback with virtual humans

ZiF'06 Proceedings of the Embodied communication in humans and machines, 2nd ZiF research group international conference on Modeling communication with robots and virtual humans
Listening heads

ZiF'06 Proceedings of the Embodied communication in humans and machines, 2nd ZiF research group international conference on Modeling communication with robots and virtual humans
Individualized gesturing outperforms average gesturing: evaluating gesture production in virtual humans

IVA'10 Proceedings of the 10th international conference on Intelligent virtual agents
Turn-taking cues in task-oriented dialogue

Computer Speech and Language
Robust classification of face and head gestures in video

Image and Vision Computing
The role of synchrony and ambiguity in speech-gesture integration during comprehension

Journal of Cognitive Neuroscience
On eye-gaze and turn-taking

Proceedings of the 2010 workshop on Eye gaze in intelligent human machine interaction
How to train your avatar: a data driven approach to gesture generation

IVA'11 Proceedings of the 10th international conference on Intelligent virtual agents
Towards a common framework for multimodal generation: the behavior markup language

IVA'06 Proceedings of the 6th international conference on Intelligent Virtual Agents
Nonverbal behavior generator for embodied conversational agents

IVA'06 Proceedings of the 6th international conference on Intelligent Virtual Agents
Implementing expressive gesture synthesis for embodied conversational agents

GW'05 Proceedings of the 6th international conference on Gesture in Human-Computer Interaction and Simulation
Systematicity and idiosyncrasy in iconic gesture use: empirical analysis and computational modeling

GW'09 Proceedings of the 8th international conference on Gesture in Embodied Communication and Human-Computer Interaction
To beat or not to beat: beat gestures in direction giving

GW'09 Proceedings of the 8th international conference on Gesture in Embodied Communication and Human-Computer Interaction
First impressions: users' judgments of virtual agents' personality and interpersonal attitude in first encounters

IVA'12 Proceedings of the 12th international conference on Intelligent Virtual Agents
Towards the automatic detection of spontaneous agreement and disagreement based on nonverbal behaviour: A survey of related cues, databases, and tools

Image and Vision Computing
Does our speech change when we cannot gesture?

Speech Communication
Analysis of relationship between head motion events and speech in dialogue conversations

Speech Communication
Handling pain: The semantic interplay of speech and co-speech hand gestures in the description of pain sensations

Speech Communication
A multimodal approach to markedness in spoken French

Speech Communication
Effects of perturbation and prosody on the coordination of speech and gesture

Speech Communication
Infants temporally coordinate gesture-speech combinations before they produce their first words

Speech Communication
Tracking eyebrows and head gestures associated with spoken prosody

Speech Communication

Quantified Score

Hi-index	0.00

Visualization

Abstract

Gestures and speech interact. They are linked in language production and perception, with their interaction contributing to felicitous communication. The multifaceted nature of these interactions has attracted considerable attention from the speech and gesture community. This article provides an overview of our current understanding of manual and head gesture form and function, of the principle functional interactions between gesture and speech aiding communication, transporting meaning and producing speech. Furthermore, we present an overview of research on temporal speech-gesture synchrony, including the special role of prosody in speech-gesture alignment. In addition, we provide a summary of tools and data available for gesture analysis, and describe speech-gesture interaction models and simulations in technical systems. This overview also serves as an introduction to a Special Issue covering a wide range of articles on these topics. We provide links to the Special Issue throughout this paper.