Face and gesture-based interaction for displaying comic books

  • Authors:
  • Hang-Bong Kang;Myung-Ho Ju

  • Affiliations:
  • Dept. of Computer Eng., Catholic University of Korea, Bucheon, Gyonggi-Do, Korea;Dept. of Computer Eng., Catholic University of Korea, Bucheon, Gyonggi-Do, Korea

  • Venue:
  • PSIVT'07 Proceedings of the 2nd Pacific Rim conference on Advances in image and video technology
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this paper, we present human robot interaction techniques such as face pose and hand gesture for efficient viewing comics through the robot. For the controlling of the viewing order of the panel, we propose a robust face pose recognition method using the pose appearance manifold. We represent each pose of a person's face as connected low-dimensional appearance manifolds which are approximated by the affine plane. Then, face pose recognition is performed by computing the minimal distance from the given face image to the sub-pose manifold. To handle partially occluded faces, we generate an occlusion mask and then put the lower weights on the occluded pixels of the given image to recognize occluded face pose. For illumination variations in the face, we perform coarse normalization on skin regions using histogram equalization. To recognize hand gestures, we compute the center of gravity of the hand using skeleton algorithm and count the number of active fingers. Also, we detect index finger's moving direction. The contents in the panel are represented by the scene graph and can be updated according to the user's control. Based on the face pose and hand gesture recognition result, an audience can manipulate contents and finally appreciate the comics in his own style.