Survey on speech emotion recognition: Features, classification schemes, and databases

Authors:
Moataz El Ayadi;Mohamed S. Kamel;Fakhri Karray
Affiliations:
Engineering Mathematics and Physics, Cairo University, Giza 12613, Egypt;Electrical and Computer Engineering, University of Waterloo, 200 University Avenue W., Waterloo, Ontario, Canada N2L 1V9;Electrical and Computer Engineering, University of Waterloo, 200 University Avenue W., Waterloo, Ontario, Canada N2L 1V9
Venue:
Pattern Recognition
Year:
2011

Citing 51
Cited 20

Attention, intentions, and the structure of discourse

Computational Linguistics
Fundamentals of speech recognition

Fundamentals of speech recognition
Decision Combination in Multiple Classifier Systems

IEEE Transactions on Pattern Analysis and Machine Intelligence
ICARUS: source generator based real-time recognition of speech in noisy stressful and Lombard effect environments

Speech Communication
Bagging predictors

Machine Learning
Enhancement of noisy speech signals: application to mobile radio communications

Speech Communication
A decision-theoretic generalization of on-line learning and an application to boosting

Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
Statistical Pattern Recognition: A Review

IEEE Transactions on Pattern Analysis and Machine Intelligence
An introduction to support Vector Machines: and other kernel-based learning methods

An introduction to support Vector Machines: and other kernel-based learning methods
Toward Machine Emotional Intelligence: Analysis of Affective Physiological State

IEEE Transactions on Pattern Analysis and Machine Intelligence - Graph Algorithms and Computer Vision
A Theoretical Study on Six Classifier Fusion Strategies

IEEE Transactions on Pattern Analysis and Machine Intelligence
Unsupervised Feature Selection Using Feature Similarity

IEEE Transactions on Pattern Analysis and Machine Intelligence
Neural Networks for Pattern Recognition

Neural Networks for Pattern Recognition
Discrete Time Processing of Speech Signals

Discrete Time Processing of Speech Signals
Linear Prediction of Speech

Linear Prediction of Speech
Recognition of Affective Communicative Intent in Robot-Directed Speech

Autonomous Robots
A Tutorial on Support Vector Machines for Pattern Recognition

Data Mining and Knowledge Discovery
A Greedy EM Algorithm for Gaussian Mixture Learning

Neural Processing Letters
Baby ears: a recognition system for affective vocalizations

Speech Communication
Describing the emotional states that are expressed in speech

Speech Communication - Special issue on speech and emotion
The role of voice quality in communicating emotion, mood and attitude

Speech Communication - Special issue on speech and emotion
Emotions, speech and the ASR framework

Speech Communication - Special issue on speech and emotion
Multimodal Human Emotion/Expression Recognition

FG '98 Proceedings of the 3rd. International Conference on Face & Gesture Recognition
Object Recognition from Local Scale-Invariant Features

ICCV '99 Proceedings of the International Conference on Computer Vision-Volume 2 - Volume 2
The production and recognition of emotions in speech: features and algorithms

International Journal of Human-Computer Studies - Application of affective computing in human—Computer interaction
Combining Pattern Classifiers: Methods and Algorithms

Combining Pattern Classifiers: Methods and Algorithms
A computational theory of the function of clue words in argument understanding

ACL '84 Proceedings of the 10th International Conference on Computational Linguistics and 22nd annual meeting on Association for Computational Linguistics
A computational model for the automatic recognition of affect in speech

A computational model for the automatic recognition of affect in speech
Comparison between Fuzzy and NN Method for Speech Emotion Recognition

ICITA '05 Proceedings of the Third International Conference on Information Technology and Applications (ICITA'05) Volume 2 - Volume 02
Linear Asymmetric Classifier for cascade detectors

ICML '05 Proceedings of the 22nd international conference on Machine learning
A Dynamic Conditional Random Field Model for Foreground and Shadow Segmentation

IEEE Transactions on Pattern Analysis and Machine Intelligence
ASR for emotional speech: Clarifying the issues and enhancing performance

Neural Networks - Special issue: Emotion and brain
Hidden Markov model-based speech emotion recognition

ICME '03 Proceedings of the 2003 International Conference on Multimedia and Expo - Volume 2
Emotion detection in task-oriented spoken dialogues

ICME '03 Proceedings of the 2003 International Conference on Multimedia and Expo - Volume 3 (ICME '03) - Volume 03
Emotional Speech Analysis on Nonlinear Manifold

ICPR '06 Proceedings of the 18th International Conference on Pattern Recognition - Volume 03
Ubiquitous Intelligence and Computing: Third International Conference, UIC 2006, Wuhan, China, September 3-6, 2006, Proceedings (Lecture Notes in Computer Science)

Ubiquitous Intelligence and Computing: Third International Conference, UIC 2006, Wuhan, China, September 3-6, 2006, Proceedings (Lecture Notes in Computer Science)
Data Mining

Data Mining
Rapid and brief communication: Combining classifier decisions for robust speaker identification

Pattern Recognition
Ensemble methods for spoken emotion recognition in call-centres

Speech Communication
On the determination of optimal model order for GMM-based text-independent speaker identification

EURASIP Journal on Applied Signal Processing
Prosody and speaker state: paralinguistics, pragmatics, and proficiency

Prosody and speaker state: paralinguistics, pragmatics, and proficiency
Investigating glottal parameters for differentiating emotional categories with similar prosodics

ICASSP '09 Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing
Combining acoustic features for improved emotion recognition in mandarin speech

ACII'05 Proceedings of the First international conference on Affective Computing and Intelligent Interaction
An enhanced speech emotion recognition system based on discourse information

ICCS'06 Proceedings of the 6th international conference on Computational Science - Volume Part I
Prosody conversion from neutral speech to emotional speech

IEEE Transactions on Audio, Speech, and Language Processing
Analysis of Emotionally Salient Aspects of Fundamental Frequency for Emotion Detection

IEEE Transactions on Audio, Speech, and Language Processing
Switching Linear Dynamical Systems for Noise Robust Speech Recognition

IEEE Transactions on Audio, Speech, and Language Processing
A kurtosis-based dynamic approach to Gaussian mixture modeling

IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans
Paper: Modeling by shortest data description

Automatica (Journal of IFAC)
Hidden Markov processes

IEEE Transactions on Information Theory
Error bounds for convolutional codes and an asymptotically optimum decoding algorithm

IEEE Transactions on Information Theory

Dimensionality reduction and classification analysis on the audio section of the SEMAINE database

ACII'11 Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part II
Naturalistic affective expression classification by a multi-stage approach based on hidden Markov models

ACII'11 Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part II
Emotional speech classification using hidden conditional random fields

Proceedings of the Second Symposium on Information and Communication Technology
Classification of emotional speech using 3DEC hierarchical classifier

Speech Communication
Emotion recognition from speech: a review

International Journal of Speech Technology
Speaker-independent emotion recognition exploiting a psychologically-inspired binary cascade classification schema

International Journal of Speech Technology
A survey of methods for data fusion and system adaptation using autonomic nervous system responses in physiological computing

Interacting with Computers
Spoken Content Retrieval: A Survey of Techniques and Technologies

Foundations and Trends in Information Retrieval
Ubiquitous emotion-aware computing

Personal and Ubiquitous Computing
Cross-validation of bimodal health-related stress assessment

Personal and Ubiquitous Computing
Depression recognition based on dynamic facial and vocal expression features using partial least square regression

Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge
Audiovisual three-level fusion for continuous estimation of Russell's emotion circumplex

Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge
GPSO versus GA in facial emotion detection

International Journal of Artificial Intelligence and Soft Computing
Shape-based modeling of the fundamental frequency contour for emotion detection in speech

Computer Speech and Language
Speech emotional features extraction based on electroglottograph

Neural Computation
Hierarchical emotion classification using genetic algorithms

Proceedings of the Fourth Symposium on Information and Communication Technology
Recognizing and regulating e-learners' emotions based on interactive Chinese texts in e-learning systems

Knowledge-Based Systems
Modeling phonetic pattern variability in favor of the creation of robust emotion classifiers for real-life applications

Computer Speech and Language
Class-specific multiple classifiers scheme to recognize emotions from speech signals

Computer Speech and Language
Exploiting Psychological Factors for Interaction Style Recognition in Spoken Conversation

IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP)

Quantified Score

Hi-index	0.01

Visualization

Abstract

Recently, increasing attention has been directed to the study of the emotional content of speech signals, and hence, many systems have been proposed to identify the emotional content of a spoken utterance. This paper is a survey of speech emotion classification addressing three important aspects of the design of a speech emotion recognition system. The first one is the choice of suitable features for speech representation. The second issue is the design of an appropriate classification scheme and the third issue is the proper preparation of an emotional speech database for evaluating system performance. Conclusions about the performance and limitations of current speech emotion recognition systems are discussed in the last section of this survey. This section also suggests possible ways of improving speech emotion recognition systems.