Fully Automatic Upper Facial Action Recognition

• Authors: Ashish Kapoor;Yuan Qi;Rosalind W. Picard
• Venue: AMFG '03 Proceedings of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures
• Year: 2003

This paper provides a new fully automatic framework to analyzefacial action units, the fundamental building blocks of facialexpression enumerated in Paul Ekman's FacialAction Coding System(FACS). The action units examined in this paper include upperfacial muscle movements suchas inner eyebrow raise, eye widening,and so forth, which combine to form facial expressions. Althoughprior method shave obtained high recognition rates for recognizingfacial action units, these methods either use manuallypre-processed image sequences or require human specification offacial features; thus, they have exploited substantial humanintervention. This paper presents a fully automatic method,requiring no such human specification. The system first robustlydetects the pupils using an infrared sensitive camera equipped withinfrared LEDs. For each frame, the pupil positions are used tolocalize and normalize eye and eyebrow regions, which are analyzedusing PCA to recover parameters that relate to the shape of thefacial features.These parameters are used as input to classifiersbased on Support Vector Machines to recognize upper facialactionunits and all their possible combinations. On a completelynatural dataset with lots of head movements, pose changes andocclusions, the new framework achieved a recognition accuracy of69.3% for each individual AU and an accuracyof 62.5% for allpossible AU combinations. This framework achieves a higherrecognition accuracy on the Cohn-KanadeAU-coded facial expressiondatabase, which has been previously used to evaluate other facialaction recognition system.