Multimodal emotion classification in naturalistic user behavior

Authors:
Steffen Walter;Stefan Scherer;Martin Schels;Michael Glodek;David Hrabal;Miriam Schmidt;Ronald Böck;Kerstin Limbrecht;Harald C. Traue;Friedhelm Schwenker
Affiliations:
Medical Psychology, University of Ulm, Germany;Institute of Neural Information Processing, University of Ulm, Germany;Institute of Neural Information Processing, University of Ulm, Germany;Institute of Neural Information Processing, University of Ulm, Germany;Medical Psychology, University of Ulm, Germany;Institute of Neural Information Processing, University of Ulm, Germany;Otto von Guericke University Magdeburg, Germany;Medical Psychology, University of Ulm, Germany;Medical Psychology, University of Ulm, Germany;Institute of Neural Information Processing, University of Ulm, Germany
Venue:
HCII'11 Proceedings of the 14th international conference on Human-computer interaction: towards mobile and intelligent interaction environments - Volume Part III
Year:
2011

Citing 8
Cited 7

The nature of statistical learning theory

The nature of statistical learning theory
Affective computing

Affective computing
Classification of Time Series Utilizing Temporal and Decision Fusion

MCS '01 Proceedings of the Second International Workshop on Multiple Classifier Systems
Combining Pattern Classifiers: Methods and Algorithms

Combining Pattern Classifiers: Methods and Algorithms
Pattern Recognition and Machine Learning (Information Science and Statistics)

Pattern Recognition and Machine Learning (Information Science and Statistics)
A Novel Feature for Emotion Recognition in Voice Based Applications

ACII '07 Proceedings of the 2nd international conference on Affective Computing and Intelligent Interaction
Emotion Recognition Based on Physiological Changes in Music Listening

IEEE Transactions on Pattern Analysis and Machine Intelligence
“Good” and “bad” diversity in majority vote ensembles

MCS'10 Proceedings of the 9th international conference on Multiple Classifier Systems

Multiple classifier systems for the classificatio of audio-visual emotional states

ACII'11 Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part II
Classification of emotional states in a woz scenario exploiting labeled and unlabeled bio-physiological data

PSL'11 Proceedings of the First IAPR TC3 conference on Partially Supervised Learning
On instance selection in audio based emotion recognition

ANNPR'12 Proceedings of the 5th INNS IAPR TC 3 GIRPR conference on Artificial Neural Networks in Pattern Recognition
A companion technology for cognitive technical systems

COST'11 Proceedings of the 2011 international conference on Cognitive Behavioural Systems
Modeling users' mood state to improve human-machine-interaction

COST'11 Proceedings of the 2011 international conference on Cognitive Behavioural Systems
Dempster-Shafer theory with smoothness

IUKM'13 Proceedings of the 2013 international conference on Integrated Uncertainty in Knowledge Modelling and Decision Making
Multi classifier systems and forward backward feature selection algorithms to classify emotional coloured speech

Proceedings of the 15th ACM on International conference on multimodal interaction

Quantified Score

Hi-index	0.01

Visualization

Abstract

The design of intelligent personalized interactive systems, having knowledge about the user's state, his desires, needs and wishes, currently poses a great challenge to computer scientists. In this study we propose an information fusion approach combining acoustic, and biophysiological data, comprising multiple sensors, to classify emotional states. For this purpose a multimodal corpus has been created, where subjects undergo a controlled emotion eliciting experiment, passing several octants of the valence arousal dominance space. The temporal and decision level fusion of the multiple modalities outperforms the single modality classifiers and shows promising results.