Real-time 3D markerless multiple hand detection and tracking for human computer interaction applications

  • Authors:
  • Soumita Ghosh;Jianmin Zheng;Wenyu Chen;Jane Zhang;Yiyu Cai

  • Affiliations:
  • Nanyang Technological University;Nanyang Technological University;Nanyang Technological University;California Polytechnic State University;Nanyang Technological University

  • Venue:
  • Proceedings of the 9th ACM SIGGRAPH Conference on Virtual-Reality Continuum and its Applications in Industry
  • Year:
  • 2010

Quantified Score

Hi-index 0.02

Visualization

Abstract

In this paper we present a purely vision based implementation of markerless hand detection and tracking system which effectively detects and tracks the hand positions irrespective of hand orientation. A shape based detection algorithm using a new line approximation technique followed by an adaptive minimum distance classifier based tracking technique is implemented. The technique is very generic and can be practically used in all types of immersive and semi-immersive environments like simulations, three-dimensional (3D) games, visually assisted medical surgery and other human-computer interactive applications. The proposed technique overcomes certain limitations of glove based, handheld object based and marker based techniques for a truly immersive experience. The technique also overcomes the main limitation of previous vision based hand tracking systems by providing much larger degrees of freedom essential for effective virtual reality systems. The novelty of the technique proposed is that it separates the detection and tracking parts and therefore allows tracking of multiple hands in a 3D space for real-time interaction.