Perceptual user interfaces: multimodal interfaces that process what comes naturally
Communications of the ACM
Hi-index | 0.00 |
HandJabber is a movement-based interactive sound installation and research environment for exploring collaborative creative expression. HandJabber utilizes the emergence of semantic meanings from gesture and interpersonal behavior as a single interface for music performance by responding to an individual participant's hand and arm gestures, and reacting to the non-verbal interpersonal behavior between two participants. Within this context, we explore three major areas of non-verbal behavior, (a) metaphoric hand and arm gesture, (b) interpersonal space, and (c) body orientation. Our poster presents results from informal user and case studies.