Data Mining Spontaneous Facial Behavior with Automatic Expression Coding

Authors:
Marian Bartlett;Gwen Littlewort;Esra Vural;Kang Lee;Mujdat Cetin;Aytul Ercil;Javier Movellan
Affiliations:
Institute for Neural Computation, University of California, San Diego, La Jolla, USA CA 92093-0445;Institute for Neural Computation, University of California, San Diego, La Jolla, USA CA 92093-0445;Institute for Neural Computation, University of California, San Diego, La Jolla, USA CA 92093-0445 and Engineering and Natural Science, Sabanci University, Istanbul, Turkey;Human Development and Applied Psychology, University of Toronto, Ontario, Canada;Engineering and Natural Science, Sabanci University, Istanbul, Turkey;Engineering and Natural Science, Sabanci University, Istanbul, Turkey;Institute for Neural Computation, University of California, San Diego, La Jolla, USA CA 92093-0445
Venue:
Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction
Year:
2008

Citing 10
Cited 1

Classifying Facial Actions

IEEE Transactions on Pattern Analysis and Machine Intelligence
Probabilistic Modeling of Local Appearance and Spatial Relationships for Object Recognition

CVPR '98 Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Comprehensive Database for Facial Expression Analysis

FG '00 Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000
Robust Real-Time Face Detection

International Journal of Computer Vision
Driver Fatigue Detection Based Intelligent Vehicle Control

ICPR '06 Proceedings of the 18th International Conference on Pattern Recognition - Volume 02
Human computing and machine understanding of human behavior: a survey

Proceedings of the 8th international conference on Multimodal interfaces
A generative framework for real time object detection and classification

Computer Vision and Image Understanding - Special issue on eye detection and tracking
Task oriented facial behavior recognition with selective sensing

Computer Vision and Image Understanding
Dynamics of facial expression extracted automatically from video

Image and Vision Computing
An automated face reader for fatigue detection

FGR' 04 Proceedings of the Sixth IEEE international conference on Automatic face and gesture recognition

Fusion of fragmentary classifier decisions for affective state recognition

MPRSS'12 Proceedings of the First international conference on Multimodal Pattern Recognition of Social Signals in Human-Computer-Interaction

Quantified Score

Hi-index	0.00

Visualization

Abstract

The computer vision field has advanced to the point that we are now able to begin to apply automatic facial expression recognition systems to important research questions in behavioral science. The machine perception lab at UC San Diego has developed a system based on machine learning for fully automated detection of 30 actions from the facial action coding system (FACS). The system, called Computer Expression Recognition Toolbox (CERT), operates in real-time and is robust to the video conditions in real applications. This paper describes two experiments which are the first applications of this system to analyzing spontaneous human behavior: Automated discrimination of posed from genuine expressions of pain, and automated detection of driver drowsiness. The analysis revealed information about facial behavior during these conditions that were previously unknown, including the coupling of movements. Automated classifiers were able to differentiate real from fake pain significantly better than naïve human subjects, and to detect critical drowsiness above 98% accuracy. Issues for application of machine learning systems to facial expression analysis are discussed.