Coding, Analysis, Interpretation, and Recognition of Facial Expressions
IEEE Transactions on Pattern Analysis and Machine Intelligence
A Tutorial on Support Vector Machines for Pattern Recognition
Data Mining and Knowledge Discovery
Automatic Spoken Affect Classification and Analysis
FG '96 Proceedings of the 2nd International Conference on Automatic Face and Gesture Recognition (FG '96)
Multimodal Human Emotion/Expression Recognition
FG '98 Proceedings of the 3rd. International Conference on Face & Gesture Recognition
FG '00 Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000
Recognizing Lower Face Action Units for Facial Expression Analysis
FG '00 Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000
ICCV '95 Proceedings of the Fifth International Conference on Computer Vision
Toward multimodal fusion of affective cues
Proceedings of the 1st ACM international workshop on Human-centered multimedia
Audio-visual emotion recognition in adult attachment interview
Proceedings of the 8th international conference on Multimodal interfaces
A survey of affect recognition methods: audio, visual and spontaneous expressions
Proceedings of the 9th international conference on Multimodal interfaces
Automatic emotional stimulus identification from facial expressions
SPPR'07 Proceedings of the Fourth conference on IASTED International Conference: Signal Processing, Pattern Recognition, and Applications
The emotion expression robot through the affective interaction: KOBIE
Proceedings of the 1st international conference on Robot communication and coordination
Estimating emotion regulation capabilities
Proceedings of the 1st international conference on PErvasive Technologies Related to Assistive Environments
Emotion Recognition through Multiple Modalities: Face, Body Gesture, Speech
Affect and Emotion in Human-Computer Interaction
Decision-Level Fusion for Audio-Visual Laughter Detection
MLMI '08 Proceedings of the 5th international workshop on Machine Learning for Multimodal Interaction
ACM Transactions on Applied Perception (TAP)
Adaptive Estimation of Emotion Generation for an Ambient Agent Model
AmI '08 Proceedings of the European Conference on Ambient Intelligence
Combining Empirical Studies of Audio-Lingual and Visual-Facial Modalities for Emotion Recognition
KES '07 Knowledge-Based Intelligent Information and Engineering Systems and the XVII Italian Workshop on Neural Networks on Proceedings of the 11th International Conference
WI-IAT '08 Proceedings of the 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology - Volume 01
Emotion recognition using brain activity
CompSysTech '08 Proceedings of the 9th International Conference on Computer Systems and Technologies and Workshop for PhD Students in Computing
Evidence Theory-Based Multimodal Emotion Recognition
MMM '09 Proceedings of the 15th International Multimedia Modeling Conference on Advances in Multimedia Modeling
How people talk when teaching a robot
Proceedings of the 4th ACM/IEEE international conference on Human robot interaction
Computers in Entertainment (CIE) - SPECIAL ISSUE: Media Arts and Games (Part II)
Designing a multi-modal affective knowledge-based user interface: combining empirical studies
Proceedings of the 2008 conference on Knowledge-Based Software Engineering: Proceedings of the Eighth Joint Conference on Knowledge-Based Software Engineering
Proceedings of the 2nd Workshop on Child, Computer and Interaction
Towards adapting fantasy, curiosity and challenge in multimodal dialogue systems for preschoolers
Proceedings of the 2009 international conference on Multimodal interfaces
Multimodal inference for driver-vehicle interaction
Proceedings of the 2009 international conference on Multimodal interfaces
IEEE Transactions on Multimedia
Overview of automatic facial expressions analysis
VIIP '07 The Seventh IASTED International Conference on Visualization, Imaging and Image Processing
IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
Automatic emotional stimulus identification from facial expressions
SPPRA '07 Proceedings of the Fourth IASTED International Conference on Signal Processing, Pattern Recognition, and Applications
Enhancement of human computer interaction with facial electromyographic sensors
OZCHI '09 Proceedings of the 21st Annual Conference of the Australian Computer-Human Interaction Special Interest Group: Design: Open 24/7
PRICAI'06 Proceedings of the 9th Pacific Rim international conference on Artificial intelligence
On assisting a visual-facial affect recognition system with keyboard-stroke pattern information
Knowledge-Based Systems
An experimental study on physiological parameters toward driver emotion recognition
EHAWC'07 Proceedings of the 2007 international conference on Ergonomics and health aspects of work with computers
Combining audio and video for detection of spontaneous emotions
BioID_MultiComm'09 Proceedings of the 2009 joint COST 2101 and 2102 international conference on Biometric ID management and multimodal communication
Affect- and behaviour-related assistance for families in the home environment
Proceedings of the 3rd International Conference on PErvasive Technologies Related to Assistive Environments
Visual affect recognition
Genetic search feature selection for affective modeling: a case study on reported preferences
Proceedings of the 3rd international workshop on Affective interaction in natural environments
Object oriented architecture for affective multimodal e-learning interfaces
Intelligent Decision Technologies
Emotion recognition from EEG using higher order crossings
IEEE Transactions on Information Technology in Biomedicine - Special section on affective and pervasive computing for healthcare
Audio-visual spontaneous emotion recognition
ICMI'06/IJCAI'07 Proceedings of the ICMI 2006 and IJCAI 2007 international conference on Artifical intelligence for human computing
Proceedings of the 7th International Conference on Methods and Techniques in Behavioral Research
Emotion on the road: necessity, acceptance, and feasibility of affective computing in the car
Advances in Human-Computer Interaction - Special issue on emotion-aware natural interaction
Recognition of emotions from video using neural network models
Expert Systems with Applications: An International Journal
An affective interactive audio interface for Lovotics
Computers in Entertainment (CIE) - Theoretical and Practical Computer Applications in Entertainment
Developing and automating a prototype for assessing levels of student involvement
FAC'11 Proceedings of the 6th international conference on Foundations of augmented cognition: directing the future of adaptive systems
A real-time EEG-based BCI system for attention recognition in ubiquitous environment
Proceedings of 2011 international workshop on Ubiquitous affective awareness and intelligent interaction
CASEMANS '11 Proceedings of the 5th ACM International Workshop on Context-Awareness for Self-Managing Systems
Fusion of audio- and visual cues for real-life emotional human robot interaction
DAGM'11 Proceedings of the 33rd international conference on Pattern recognition
Hybrid fusion approach for detecting affects from multichannel physiology
ACII'11 Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part I
Audio visual emotion recognition based on triple-stream dynamic bayesian network models
ACII'11 Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part I
Adaptive facial expression recognition using inter-modal top-down context
ICMI '11 Proceedings of the 13th international conference on multimodal interfaces
Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium
Human face analysis: from identity to emotion and intention recognition
ICEB'10 Proceedings of the Third international conference on Ethics and Policy of Biometrics and International Data Sharing
AutoSelect: What You Want Is What You Get: Real-Time Processing of Visual Attention and Affect
PIT'06 Proceedings of the 2006 international tutorial and research conference on Perception and Interactive Technologies
Emotion recognition using physiological and speech signal in short-term observation
PIT'06 Proceedings of the 2006 international tutorial and research conference on Perception and Interactive Technologies
A computational model based on Gross' emotion regulation theory
Cognitive Systems Research
Multimodal object oriented user interfaces in mobile affective interaction
Multimedia Tools and Applications
Proceedings of the 14th ACM international conference on Multimodal interaction
Proceedings of the 14th ACM international conference on Multimodal interaction
Demonstration of IlluMe: creating ambient according to instant message logs
ACL '12 Proceedings of the ACL 2012 System Demonstrations
Modeling users' mood state to improve human-machine-interaction
COST'11 Proceedings of the 2011 international conference on Cognitive Behavioural Systems
Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge
The whisper pillow: a study of technology-mediated emotional expression in close relationships
Proceedings of the 6th International Conference on Designing Pleasurable Products and Interfaces
Multiple kernel learning for emotion recognition in the wild
Proceedings of the 15th ACM on International conference on multimodal interaction
Emotion recognition using facial and audio features
Proceedings of the 15th ACM on International conference on multimodal interaction
Computer Speech and Language
Multimodal Approach for Emotion Recognition Using a Formal Computational Model
International Journal of Applied Evolutionary Computation
Development process of an affective bi-modal Intelligent Tutoring System
Intelligent Decision Technologies
Hi-index | 0.00 |
The interaction between human beings and computers will be more natural if computers are able to perceive and respond to human non-verbal communication such as emotions. Although several approaches have been proposed to recognize human emotions based on facial expressions or speech, relatively limited work has been done to fuse these two, and other, modalities to improve the accuracy and robustness of the emotion recognition system. This paper analyzes the strengths and the limitations of systems based only on facial expressions or acoustic information. It also discusses two approaches used to fuse these two modalities: decision level and feature level integration. Using a database recorded from an actress, four emotions were classified: sadness, anger, happiness, and neutral state. By the use of markers on her face, detailed facial motions were captured with motion capture, in conjunction with simultaneous speech recordings. The results reveal that the system based on facial expression gave better performance than the system based on just acoustic information for the emotions considered. Results also show the complementarily of the two modalities and that when these two modalities are fused, the performance and the robustness of the emotion recognition system improve measurably.