Affective multimodal human-computer interaction

Authors:
Maja Pantic;Nicu Sebe;Jeffrey F. Cohn;Thomas Huang
Affiliations:
Delft University of Technology, The Netherlands;University of Amsterdam, The Netherlands;University of Pittsburgh, Pittsburgh, PA;University of Illinois at Urbana-Champaign, Urbana, IL
Venue:
Proceedings of the 13th annual ACM international conference on Multimedia
Year:
2005

Citing 17
Cited 50

Affective computing

Affective computing
Bimodal expression of emotion by face and voice

MULTIMEDIA '98 Proceedings of the sixth ACM international conference on Multimedia: Face/gesture recognition and their applications
Looking at People: Sensing for Ubiquitous and Wearable Computing

IEEE Transactions on Pattern Analysis and Machine Intelligence
Toward Machine Emotional Intelligence: Analysis of Affective Physiological State

IEEE Transactions on Pattern Analysis and Machine Intelligence - Graph Algorithms and Computer Vision
Embodied contextual agent in information delivering application

Proceedings of the first international joint conference on Autonomous agents and multiagent systems: part 2
Comprehensive Database for Facial Expression Analysis

FG '00 Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000
Bimodal Emotion Recognition

FG '00 Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000
Audio-Visual Speaker Detection Using Dynamic Bayesian Networks

FG '00 Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000
Joint processing of audio-visual information for the recognition of emotional expressions in human-computer interaction

Joint processing of audio-visual information for the recognition of emotional expressions in human-computer interaction
Facial expression recognition from video sequences: temporal and static modeling

Computer Vision and Image Understanding - Special issue on Face recognition
Has the Internet become indispensable?

Communications of the ACM - Has the Internet become indispensable?
Socially Aware Computation and Communication

Computer
Human-centered design considered harmful

interactions - Ambient intelligence: exploring our living environment
Audio-visual based emotion recognition-a new approach

CVPR'04 Proceedings of the 2004 IEEE computer society conference on Computer vision and pattern recognition
Authentic facial expression analysis

FGR' 04 Proceedings of the Sixth IEEE international conference on Automatic face and gesture recognition
Emotion analysis in man-machine interaction systems

MLMI'04 Proceedings of the First international conference on Machine Learning for Multimodal Interaction
A survey on visual surveillance of object motion and behaviors

IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews

Audio-visual emotion recognition in adult attachment interview

Proceedings of the 8th international conference on Multimodal interfaces
Modeling naturalistic affective states via facial and vocal expressions recognition

Proceedings of the 8th international conference on Multimodal interfaces
Gaze-X: adaptive affective multimodal interface for single-user office scenarios

Proceedings of the 8th international conference on Multimodal interfaces
Foundations of human computing: facial expression and emotion

Proceedings of the 8th international conference on Multimodal interfaces
Editorial: The emerging roles of performance within HCI and interaction design

Interacting with Computers
Observer annotation of affective display and evaluation of expressivity: face vs. face-and-body

VisHCI '06 Proceedings of the HCSNet workshop on Use of vision in human-computer interaction - Volume 56
Bi-modal emotion recognition from expressive face and body gestures

Journal of Network and Computer Applications
Multimodal human-computer interaction: A survey

Computer Vision and Image Understanding
Affective multimodal mirror: sensing and eliciting laughter

Proceedings of the international workshop on Human-centered multimedia
Visual inference of human emotion and behaviour

Proceedings of the 9th international conference on Multimodal interfaces
How to distinguish posed from spontaneous smiles using geometric features

Proceedings of the 9th international conference on Multimodal interfaces
A survey of affect recognition methods: audio, visual and spontaneous expressions

Proceedings of the 9th international conference on Multimodal interfaces
Bimodal person-dependent emotion recognition comparison of feature level and decision level information fusion

Proceedings of the 1st international conference on PErvasive Technologies Related to Assistive Environments
User and context adaptive neural networks for emotion recognition

Neurocomputing
Towards Knowledge-Based Affective Interaction: Situational Interpretation of Affect

ACII '07 Proceedings of the 2nd international conference on Affective Computing and Intelligent Interaction
Emotion Recognition through Multiple Modalities: Face, Body Gesture, Speech

Affect and Emotion in Human-Computer Interaction
Interaction design and experiential factors: a novel case study on digital pen and paper

Mobility '08 Proceedings of the International Conference on Mobile Technology, Applications, and Systems
Boosting selection of speech related features to improve performance of multi-class SVMs in emotion detection

Expert Systems with Applications: An International Journal
Using facial expressions and peripheral physiological signals as implicit indicators of topical relevance

MM '09 Proceedings of the 17th ACM international conference on Multimedia
Three Philosophers and a Design for Expressive Gestural Interaction

Proceedings of the 2009 conference on New Trends in Software Methodologies, Tools and Techniques: Proceedings of the Eighth SoMeT_09
Emotion recognition using facial expressions with active appearance models

HCI '08 Proceedings of the Third IASTED International Conference on Human Computer Interaction
Multimodal interfaces: Challenges and perspectives

Journal of Ambient Intelligence and Smart Environments
Recognizing affect from non-stylized body motion using shape of Gaussian descriptors

Proceedings of the 2010 ACM Symposium on Applied Computing
New frameworks to boost feature selection algorithms in emotion detection for improved human-computer interaction

BVAI'07 Proceedings of the 2nd international conference on Advances in brain, vision and artificial intelligence
Interacting with context

QuaCon'09 Proceedings of the 1st international conference on Quality of context
Guest editorial: special section on new and emerging technologies in bioinformatics and bioengineering

IEEE Transactions on Information Technology in Biomedicine - Special section on new and emerging technologies in bioinformatics and bioengineering
Toward emotion aware computing: an integrated approach using multichannel neurophysiological recordings and affective visual stimuli

IEEE Transactions on Information Technology in Biomedicine - Special section on new and emerging technologies in bioinformatics and bioengineering
A comparison of general vs personalised affective models for the prediction of topical relevance

Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
Segmenting into adequate units for automatic recognition of emotion-related episodes: a speech-based approach

Advances in Human-Computer Interaction - Special issue on emotion-aware natural interaction
eMotion: un outil pour personnaliser la reconnaissance d'émotions

Proceedings of the Ergonomie et Informatique Avancee Conference
Foundations of human computing: facial expression and emotion

ICMI'06/IJCAI'07 Proceedings of the ICMI 2006 and IJCAI 2007 international conference on Artifical intelligence for human computing
Audio-visual spontaneous emotion recognition

ICMI'06/IJCAI'07 Proceedings of the ICMI 2006 and IJCAI 2007 international conference on Artifical intelligence for human computing
Modeling naturalistic affective states via facial, vocal, and bodily expressions recognition

ICMI'06/IJCAI'07 Proceedings of the ICMI 2006 and IJCAI 2007 international conference on Artifical intelligence for human computing
Gaze-X: adaptive, affective, multimodal interface for single-user office scenarios

ICMI'06/IJCAI'07 Proceedings of the ICMI 2006 and IJCAI 2007 international conference on Artifical intelligence for human computing
Reconnaissance d'Emotions: un point de vue interaction multimodale

Conference Internationale Francophone sur I'Interaction Homme-Machine
Theories, methods and current research on emotions in library and information science, information retrieval and human-computer interaction

Information Processing and Management: an International Journal
Emotion recognition using a hierarchical binary decision tree approach

Speech Communication
Guess what? a game for affective annotation of video using crowd sourcing

ACII'11 Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part I
Modeling latent discriminative dynamic of multi-dimensional affective signals

ACII'11 Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part II
Multimodal human computer interaction: a survey

ICCV'05 Proceedings of the 2005 international conference on Computer Vision in Human-Computer Interaction
Appearance manifold of facial expression

ICCV'05 Proceedings of the 2005 international conference on Computer Vision in Human-Computer Interaction
Face for ambient interface

Ambient Intelligence in Everyday Life
Vision-Based game interface using human gesture

PSIVT'06 Proceedings of the First Pacific Rim conference on Advances in Image and Video Technology
Speaker-independent emotion recognition exploiting a psychologically-inspired binary cascade classification schema

International Journal of Speech Technology
Multimodal interfaces: Challenges and perspectives

Journal of Ambient Intelligence and Smart Environments
Towards the improvement of self-service systems via emotional virtual agents

BCS-HCI '12 Proceedings of the 26th Annual BCS Interaction Specialist Group Conference on People and Computers
AffectButton: A method for reliable and valid affective self-report

International Journal of Human-Computer Studies
Competitive affective gaming: winning with a smile

Proceedings of the 21st ACM international conference on Multimedia
NovaEmötions: winning with a smile

Proceedings of the 21st ACM international conference on Multimedia
Exploiting unconscious user signals in multimodal human-computer interaction

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP) - Special Sections on the 20th Anniversary of ACM International Conference on Multimedia, Best Papers of ACM Multimedia 2012

Quantified Score

Hi-index	0.00

Visualization

Abstract

Social and emotional intelligence are aspects of human intelligence that have been argued to be better predictors than IQ for measuring aspects of success in life, especially in social interactions, learning, and adapting to what is important. When it comes to machines, not all of them will need such skills. Yet to have machines like computers, broadcast systems, and cars, capable of adapting to their users and of anticipating their wishes, endowing them with the ability to recognize user's affective states is necessary. This article discusses the components of human affect, how they might be integrated into computers, and how far are we from realizing affective multimodal human-computer interaction.