Real-time fingertip tracking and detection using Kinect depth sensor for a new writing-in-the air system

Authors:
Ziyong Feng;Shaojie Xu;Xin Zhang;Lianwen Jin;Zhichao Ye;Weixin Yang
Affiliations:
South China University of Technology, Guangzhou, China;South China University of Technology, Guangzhou, China;South China University of Technology, Guangzhou, China;South China University of Technology, Guangzhou, China;South China University of Technology, Guangzhou, China;South China University of Technology, Guangzhou, China
Venue:
Proceedings of the 4th International Conference on Internet Multimedia Computing and Service
Year:
2012

Citing 8
Cited 0

Finger Track - A Robust and Real-Time Gesture Interface

AI '97 Proceedings of the 10th Australian Joint Conference on Artificial Intelligence: Advanced Topics in Artificial Intelligence
A Virtual 3D Blackboard: 3D Finger Tracking Using a Single Camera

FG '00 Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000
Visual touchpad: a two-handed gestural input device

Proceedings of the 6th international conference on Multimodal interfaces
Real-time hand posture recognition using range data

Image and Vision Computing
Color Based Hand and Finger Detection Technology for User Interaction

ICHIT '08 Proceedings of the 2008 International Conference on Convergence and Hybrid Information Technology
Real-time head pose estimation using random regression forests

CCBR'11 Proceedings of the 6th Chinese conference on Biometric recognition
Real-time human pose recognition in parts from single depth images

CVPR '11 Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition
A real-time multi-cue hand tracking algorithm based on computer vision

VR '10 Proceedings of the 2010 IEEE Virtual Reality Conference

Quantified Score

Hi-index	0.00

Visualization

Abstract

We propose a real-time finger writing character recognition system using depth information. This system allows user to input characters by writing freely in the air with the Kinect. During the writing process, it is reasonable to assume that the finger and hand are always holding in front of torso. Firstly, we compute the depth histogram of human body and use a switch mixture Gaussian model to characterize it. Since the hand is closer to camera, a model-based threshold can segment the hand-related region out. Then, we employ an unsupervised clustering algorithm, K-means, to classify the segmented region into two parts, the finger-hand part and hand-arm part. By identifying the arm direction, we can determine the finger-hand cluster and locate the fingertip as the farthest point from the other cluster. We collected over 8000 frames writing-in-the-air sequences including two different subjects writing numbers, strokes, pattern, English and Chinese characters from two different distances. From our experiments, the proposed algorithm can provide robust and accurate fingertip detection, and achieve encouraging character recognition result.