Recognition of emotions from video using neural network models

  • Authors:
  • K. Sreenivasa Rao;V. K. Saroj;Sudhamay Maity;Shashidhar G. Koolagudi

  • Affiliations:
  • School of Information Technology, Indian Institute of Technology Kharagpur, Kharagpur 721302, West Bengal, India;School of Information Technology, Indian Institute of Technology Kharagpur, Kharagpur 721302, West Bengal, India;School of Information Technology, Indian Institute of Technology Kharagpur, Kharagpur 721302, West Bengal, India;School of Information Technology, Indian Institute of Technology Kharagpur, Kharagpur 721302, West Bengal, India

  • Venue:
  • Expert Systems with Applications: An International Journal
  • Year:
  • 2011

Quantified Score

Hi-index 12.05

Visualization

Abstract

In this paper, facial features from the video sequence are explored for characterizing the emotions. The emotions considered for this study are Anger, Fear, Happy, Sad and Neutral. For carrying out the proposed emotion recognition study, the required video data is collected from the studio, Center for Education Technology (CET), at Indian Institute of Technology (IIT) Kharagpur. The dynamic nature of the grey values of the pixels within the eye and mouth regions are used as the features to capture the emotion specific knowledge from the facial expressions. Multiscale morphological erosion and dilation operations are used to extract features from eye and mouth regions, respectively. The features extracted from left eye, right eye and mouth regions are used to develop the separate models for each emotion category. Autoassociative neural network (AANN) models are used to capture the distribution of the extracted features. The developed models are validated using subject dependent and independent emotion recognition studies. The overall performance of the proposed emotion recognition system is observed to be about 87%.