Evaluation of multimodal input for entering mathematical equations on the computer

  • Authors:
  • Lisa Anthony;Jie Yang;Kenneth R. Koedinger

  • Affiliations:
  • Carnegie Mellon University, Pittsburgh, PA;Carnegie Mellon University, Pittsburgh, PA;Carnegie Mellon University, Pittsburgh, PA

  • Venue:
  • CHI '05 Extended Abstracts on Human Factors in Computing Systems
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

Current standard interfaces for entering mathematical equations on computers are arguably limited and cumbersome. Mathematics notations have evolved to aid visual thinking and yet text-based interfaces relying on keyboard-and-mouse input do not take advantage of the natural two-dimensional aspects of math. Due to its similarities to paper-based mathematics, pen-based handwriting input may be faster, more efficient, and more preferable for entering mathematics on computers. This paper presents an empirical study that tests this hypothesis. We also explored a multimodal input method combining handwriting and speech because we hypothesize that it may enhance computer recognition and aid user cognition. Novice users were indeed faster, more efficient and enjoyed the handwriting modality more than a standard keyboard-and-mouse mathematics interface, especially as equation length and complexity increased. The multimodal handwriting-plus-speech method was faster and better liked than the keyboard-and-mouse method and was not much worse than handwriting alone.