6D hands: markerless hand-tracking for computer aided design

  • Authors:
  • Robert Wang;Sylvain Paris;Jovan Popović

  • Affiliations:
  • 3Gear Systems, Foster City, CA, USA;Adobe Systems, Cambridge, MA, USA;Adobe Systems, Seattle, WA, USA

  • Venue:
  • Proceedings of the 24th annual ACM symposium on User interface software and technology
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Computer Aided Design (CAD) typically involves tasks such as adjusting the camera perspective and assembling pieces in free space that require specifying 6 degrees of freedom (DOF). The standard approach is to factor these DOFs into 2D subspaces that are mapped to the x and y axes of a mouse. This metaphor is inherently modal because one needs to switch between subspaces, and disconnects the input space from the modeling space. In this paper, we propose a bimanual hand tracking system that provides physically-motivated 6-DOF control for 3D assembly. First, we discuss a set of principles that guide the design of our precise, easy-to-use, and comfortable-to-use system. Based on these guidelines, we describe a 3D input metaphor that supports constraint specification classically used in CAD software, is based on only a few simple gestures, lets users rest their elbows on their desk, and works alongside the keyboard and mouse. Our approach uses two consumer-grade webcams to observe the user's hands. We solve the pose estimation problem with efficient queries of a precomputed database that relates hand silhouettes to their 3D configuration. We demonstrate efficient 3D mechanical assembly of several CAD models using our hand-tracking system.