Finger and hand detection for multi-touch interfaces based on maximally stable extremal regions

  • Authors:

  • Philipp Ewerling;Alexander Kulik;Bernd Froehlich

  • Affiliations:

  • Bauhaus-Universität Weimar, Weimar, Germany;Bauhaus-Universität Weimar, Weimar, Germany;Bauhaus-Universität Weimar, Weimar, Germany

  • Venue:
  • Proceedings of the 2012 ACM international conference on Interactive tabletops and surfaces
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

We propose a new approach for touch detection on optical multi-touch devices that exploits the fact that the camera images reveal not only the actual touch points, but also objects above the screen such as the hand or arm of a user. Our touch processing relies on the Maximally Stable Extremal Regions algorithm for finding the users' fingertips in the camera image. The hierarchical structure of the generated extremal regions serves as a starting point for agglomerative clustering of the fingertips into hands. Furthermore, we suggest a heuristic supporting the identification of individual fingers as well as the distinction between left hands and right hands if all five fingers of a hand are in contact with the touch surface. Our evaluation confirmed that the system is robust against detection errors resulting from non-uniform illumination and reliably assigns touch points to individual hands based on the implicitly tracked context information. The efficient multithreaded implementation handles two-handed input from multiple users in real-time.