Real-Time 3-D Hand Posture Estimation Based on 2-D Appearance Retrieval Using Monocular Camera

  • Authors:
  • Nobutaka Shimada;Kousuke Kimura;Yoshiaki Shirai

  • Affiliations:
  • -;-;-

  • Venue:
  • RATFG-RTS '01 Proceedings of the IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems (RATFG-RTS'01)
  • Year:
  • 2001

Quantified Score

Hi-index 0.00

Visualization

Abstract

Abstract: This paper proposes a system estimating arbitrary 3-D human hand postures in real-time. Differently from most of the previous real-time system, it can accept not only pre-determined hand signs but also arbitrary postures and it works in a monocular camera environment. The estimation is based on a 2-D image retrieval. More than 16000 possible hand appearances (silhouette contours) are first generated from a given 3-D shape model by rotating model joints and stored in an appearance database. Every appearance is tagged with its own joint angles which are used when the appearance was generated. By retrieving the appearance in the database well-matching to the input image contour, the joint angles of the input shape can be rapidly obtained. For robust matching, little difference between the appearances and the input is adjusted before matching considering shape deformation and quantization error in appearence samping. In order to achieve the real-time processing, the search area is reduced by using an adjacency map in the database. In the map, adjacent appearances having similar joint angles are connected with each other. For each appearance, a neighborhood which consists of the adjacent appearances is defined. In each frame, the search area is limited to the neighborhood of the estimated appearance in the previous frame. The best candidate at one frame is sometimes wrong due to approximation errors, too rapid motions or ambiguities caused by self-occlusion. To prevent tracking failures, a fixed number of the well-matching appearances are saved at every frame. After the multiple neighborhoods of the saved appearances are merged, the unified neighborhood is searched for the estimate efficiently by Beam search. These algorithms are implemented on a PC cluster system consisting of 6 PCs (Pentium III 600MHz) and real-time processing is achieved. The resulted posture estimates are shown by experimental examples.