Enhancing expression recognition in the wild with unlabeled reference data

Authors:
Mengyi Liu;Shaoxin Li;Shiguang Shan;Xilin Chen
Affiliations:
Key Lab of Intelligent Information Processing, Chinese Academy of Sciences (CAS), Beijing, China,Institute of Computing Technology, CAS, Beijing, China,Graduate University of Chinese Academy of Sc ...;Key Lab of Intelligent Information Processing, Chinese Academy of Sciences (CAS), Beijing, China,Institute of Computing Technology, CAS, Beijing, China,Graduate University of Chinese Academy of Sc ...;Key Lab of Intelligent Information Processing, Chinese Academy of Sciences (CAS), Beijing, China,Institute of Computing Technology, CAS, Beijing, China;Key Lab of Intelligent Information Processing, Chinese Academy of Sciences (CAS), Beijing, China,Institute of Computing Technology, CAS, Beijing, China
Venue:
ACCV'12 Proceedings of the 11th Asian conference on Computer Vision - Volume Part II
Year:
2012

Citing 20
Cited 1

Automatic Analysis of Facial Expressions: The State of the Art

IEEE Transactions on Pattern Analysis and Machine Intelligence
Think globally, fit locally: unsupervised learning of low dimensional manifolds

The Journal of Machine Learning Research
Manifold of Facial Expression

AMFG '03 Proceedings of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures
Semi-supervised learning for facial expression recognition

MIR '03 Proceedings of the 5th ACM SIGMM international workshop on Multimedia information retrieval
Histograms of Oriented Gradients for Human Detection

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1 - Volume 01
Semi-supervised graph clustering: a kernel approach

ICML '05 Proceedings of the 22nd international conference on Machine learning
Non-negative matrix factorization for semi-supervised data clustering

Knowledge and Information Systems
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions

IEEE Transactions on Pattern Analysis and Machine Intelligence
Facial expression recognition based on Local Binary Patterns: A comprehensive study

Image and Vision Computing
Toward Practical Smile Detection

IEEE Transactions on Pattern Analysis and Machine Intelligence
Facial expression recognition on multiple manifolds

Pattern Recognition
Semi-supervised facial expressions annotation using co-training with fast probabilistic tri-class SVMs

ICANN'10 Proceedings of the 20th international conference on Artificial neural networks: Part II
LIBSVM: A library for support vector machines

ACM Transactions on Intelligent Systems and Technology (TIST)
Learning bayesian network classifiers for facial expression recognition using both labeled and unlabeled data

CVPR'03 Proceedings of the 2003 IEEE computer society conference on Computer vision and pattern recognition
Appearance manifold of facial expression

ICCV'05 Proceedings of the 2005 international conference on Computer Vision in Human-Computer Interaction
An associate-predict model for face recognition

CVPR '11 Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition
Smile Detection by Boosting Pixel Differences

IEEE Transactions on Image Processing
Improved facial expression recognition via uni-hyperplane classification

CVPR '12 Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Learning active facial patches for expression analysis

CVPR '12 Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Pose, illumination and expression invariant pairwise face-similarity measure via Doppelgänger list comparison

ICCV '11 Proceedings of the 2011 International Conference on Computer Vision

Facial expression recognition experiments with data from television broadcasts and the World Wide Web

Image and Vision Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Facial expression recognition is an important task in human-computer interaction. Some methods work well on "lab-controlled" data. However, their performances degenerate dramatically on real-world data as expression covers large variations, including pose, illumination, occlusion, and even culture change. To deal with this problem, large scale data is definitely needed. On the other hand, collecting and labeling wild expression data can be difficult and time consuming. In this paper, aiming at robust expression recognition in wild which suffers from the mentioned problems, we propose a semi-supervised method to make use of the large scale unlabeled data in two steps: 1) We enrich reference manifolds using selected unlabeled data which are closed to certain kind of expression. The learned manifolds can help smooth the variation of original data and provide reliable metric to maintain semantic similarity of expression; 2) To elevate the original labeled set for enhanced training, we iteratively employ the semi-supervised clustering to assign labels for unlabeled data and add the most discriminant ones into the labeled set. Experiments on the latest wild expression database SFEW and GENKI show that the proposed method can effectively exploit unlabeled data to improve the performance on real-world expression recognition.