Multidimensional gesture sensing at the piano keyboard

  • Authors:
  • Andrew P. McPherson;Youngmoo E. Kim

  • Affiliations:
  • Drexel University, Philadelphia, Pennsylvania, USA;Drexel University, Philadelphia, Pennsylvania, USA

  • Venue:
  • Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
  • Year:
  • 2011

Quantified Score

Hi-index 0.01

Visualization

Abstract

In this paper we present a new keyboard interface for computer music applications. Where traditional keyboard controllers report the velocity of each key-press, our interface senses up to five separate dimensions: velocity, percussiveness, rigidity, weight, and depth. These dimensions, which we identified based on the pedagogical piano literature and pilot studies with professional pianists, together present a rich picture of physical gestures at the keyboard, including information on the performer's motion before, during, and after a note is played. User studies confirm that the sensed dimensions are intuitive and controllable and that mappings between gesture and sound produce novel, playable musical instruments, even for users without prior keyboard experience. The multidimensional sensing capability demonstrated in this paper is also potentially applicable to button interfaces outside the musical domain.