Vision-based handwriting recognition for unrestricted text input in mid-air

Authors:
Alexander Schick;Daniel Morlock;Christoph Amma;Tanja Schultz;Rainer Stiefelhagen
Affiliations:
Fraunhofer IOSB, Karlsruhe, Germany;Fraunhofer IOSB & Karlsruhe Institute of Technology, Karlsruhe, Germany;Karlsruhe Institute of Technology, Karlsruhe, Germany;Karlsruhe Institute of Technology, Karlsruhe, Germany;Fraunhofer IOSB & Karlsruhe Institute of Technology, Karlsruhe, Germany
Venue:
Proceedings of the 14th ACM international conference on Multimodal interaction
Year:
2012

Citing 10
Cited 1

On-Line and Off-Line Handwriting Recognition: A Comprehensive Survey

IEEE Transactions on Pattern Analysis and Machine Intelligence
Real-Time Self-Calibrating Stereo Person Tracking Using 3-D Shape Estimation from Blob Features

ICPR '96 Proceedings of the International Conference on Pattern Recognition (ICPR '96) Volume III-Volume 7276 - Volume 7276
Pointing gesture recognition based on 3D-tracking of face, hands and head orientation

Proceedings of the 5th international conference on Multimodal interfaces
Hand-Gesture Based Text Input for Wearable Computers

ICVS '06 Proceedings of the Fourth IEEE International Conference on Computer Vision Systems
Mid-air text input techniques for very large wall displays

Proceedings of Graphics Interface 2009
Extending touch: towards interaction with large-scale surfaces

Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces
American sign language recognition with the kinect

ICMI '11 Proceedings of the 13th international conference on multimodal interfaces
3D space handwriting recognition with ligature model

UCS'06 Proceedings of the Third international conference on Ubiquitous Computing Systems
Gesture Recognition: A Survey

IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews
Airwriting: Hands-Free Mobile Text Input by Spotting and Continuous Recognition of 3d-Space Handwriting with Inertial Sensors

ISWC '12 Proceedings of the 2012 16th Annual International Symposium on Wearable Computers (ISWC)

SpeeG2: a speech- and gesture-based interface for efficient controller-free text input

Proceedings of the 15th ACM on International conference on multimodal interaction

Quantified Score

Hi-index	0.00

Visualization

Abstract

We propose a vision-based system that recognizes handwriting in mid-air. The system does not depend on sensors or markers attached to the users and allows unrestricted character and word input from any position. It is the result of combining handwriting recognition based on Hidden Markov Models with multi-camera 3D hand tracking. We evaluated the system for both quantitative and qualitative aspects. The system achieves recognition rates of 86.15% for character and 97.54% for small-vocabulary isolated word recognition. Limitations are due to slow and low-resolution cameras or physical strain. Overall, the proposed handwriting recognition system provides an easy-to-use and accurate text input modality without placing restrictions on the users.