Model-based hand pose estimation via spatial-temporal hand parsing and 3D fingertip localization

  • Authors:
  • Hui Liang;Junsong Yuan;Daniel Thalmann;Zhengyou Zhang

  • Affiliations:
  • Institute for Media Innovation & School of EEE, Nanyang Technological University, Singapore, Singapore 637553;School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, Singapore 639798;Institute for Media Innovation, Nanyang Technological University, Singapore, Singapore 637553;Microsoft Research, Redmond, USA 98052

  • Venue:
  • The Visual Computer: International Journal of Computer Graphics
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this paper we present a novel vision-based markerless hand pose estimation scheme with the input of depth image sequences. The proposed scheme exploits both temporal constraints and spatial features of the input sequence, and focuses on hand parsing and 3D fingertip localization for hand pose estimation. The hand parsing algorithm incorporates a novel spatial-temporal feature into a Bayesian inference framework to assign the correct label to each image pixel. The 3D fingertip localization algorithm adapts a recently developed geodesic extrema extraction method to fingertip detection with the hand parsing algorithm, a novel path-reweighting method and K-means clustering in metric space. The detected 3D fingertip locations are finally used for hand pose estimation with an inverse kinematics solver. Quantitative experiments on synthetic data show the proposed hand pose estimation scheme can accurately capture the natural hand motion. A simulated water-oscillator application is also built to demonstrate the effectiveness of the proposed method in human-computer interaction scenarios.