Exploring bag of words architectures in the facial expression domain

Authors:
Karan Sikka;Tingfan Wu;Josh Susskind;Marian Bartlett
Affiliations:
Machine Perception Laboratory, University of California San Diego;Machine Perception Laboratory, University of California San Diego;Machine Perception Laboratory, University of California San Diego;Machine Perception Laboratory, University of California San Diego
Venue:
ECCV'12 Proceedings of the 12th international conference on Computer Vision - Volume 2
Year:
2012

Citing 14
Cited 1

Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns

IEEE Transactions on Pattern Analysis and Machine Intelligence
Robust Real-Time Face Detection

International Journal of Computer Vision
Creating Efficient Codebooks for Visual Recognition

ICCV '05 Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 - Volume 01
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories

CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2
A Visual Vocabulary for Flower Classification

CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2
Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study

CVPRW '06 Proceedings of the 2006 Conference on Computer Vision and Pattern Recognition Workshop
Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions

IEEE Transactions on Pattern Analysis and Machine Intelligence
Kernel Codebooks for Scene Categorization

ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part III
Facial expression recognition based on Local Binary Patterns: A comprehensive study

Image and Vision Computing
Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations

ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Facial-component-based bag of words and PHOG descriptor for facial expression recognition

SMC'09 Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics
Vlfeat: an open and portable library of computer vision algorithms

Proceedings of the international conference on Multimedia
Sampling strategies for bag-of-features image classification

ECCV'06 Proceedings of the 9th European conference on Computer Vision - Volume Part IV
Cats and dogs

CVPR '12 Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Multiple kernel learning for emotion recognition in the wild

Proceedings of the 15th ACM on International conference on multimodal interaction

Quantified Score

Hi-index	0.00

Visualization

Abstract

Automatic facial expression recognition (AFER) has undergone substantial advancement over the past two decades. This work explores the application of bag of words (BoW), a highly matured approach for object and scene recognition to AFER. We proceed by first highlighting the reasons that makes the task for BoW differ for AFER compared to object and scene recognition. We propose suitable extensions to BoW architecture for the AFER's task. These extensions are able to address some of the limitations of current state of the art appearance-based approaches to AFER. Our BoW architecture is based on the spatial pyramid framework, augmented by multiscale dense SIFT features, and a recently proposed approach for object classification: locality-constrained linear coding and max-pooling. Combining these, we are able to achieve a powerful facial representation that works well even with linear classifiers. We show that a well designed BoW architecture can provide a performance benefit for AFER, and elements of the proposed BoW architecture are empirically evaluated. The proposed BoW approach supersedes previous state of the art results by achieving an average recognition rate of 96% on AFER for two public datasets.