All smiles: automatic photo enhancement by facial expression analysis

  • Authors:
  • Rajvi Shah;Vivek Kwatra

  • Affiliations:
  • IIIT Hyderabad, India and Google Research, Mountain View, CA;Google Research, Mountain View, CA

  • Venue:
  • Proceedings of the 9th European Conference on Visual Media Production
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

We propose a framework for automatic enhancement of group photographs by facial expression analysis. We are motivated by the observation that group photographs are seldom perfect. Subjects may have inadvertently closed their eyes, may be looking away, or may not be smiling at that moment. Given a set of photographs of the same group of people, our algorithm uses facial analysis to determine a goodness score for each face instance in those photos. This scoring function is based on classifiers for facial expressions such as smiles and eye-closure, trained over a large set of annotated photos. Given these scores, a best composite for the set is synthesized by (a) selecting the photo with the best overall score, and (b) replacing any low-scoring faces in that photo with high-scoring faces of the same person from other photos, using alignment and seamless composition.