Improving accuracy in back-of-device multitouch typing: a clustering-based approach to keyboard updating

  • Authors:
  • Daniel Buschek;Oliver Schoenleben;Antti Oulasvirta

  • Affiliations:
  • Helsinki Institute for Information Technology, Helsinki, Finland;Helsinki Institute for Information Technology, Helsinki, Finland;Max Planck Institute for Informatics, Saarbruecken, Germany

  • Venue:
  • Proceedings of the 19th international conference on Intelligent User Interfaces
  • Year:
  • 2014

Quantified Score

Hi-index 0.00

Visualization

Abstract

Recent work has shown that a multitouch sensor attached to the back of a handheld device can allow rapid typing engaging all ten fingers. However, high error rates remain a problem, because the user can not see or feel key-targets on the back. We propose a machine learning approach that can significantly improve accuracy. The method considers hand anatomy and movement ranges of fingers. The key insight is a combination of keyboard and hand models in a hierarchical clustering method. This enables dynamic re-estimation of key-locations while typing to account for changes in hand postures and movement ranges of fingers. We also show that accuracy can be further improved with language models. Results from a user study show improvements of over 40% compared to the previously deployed "naive" approach. We examine entropy as a touch precision metric with respect to typing experience. We also find that the QWERTY layout is not ideal. Finally, we conclude with ideas for further improvements.