A generic platform for addressing the multimodal challenge
CHI '95 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Component software: beyond object-oriented programming
Component software: beyond object-oriented programming
Maintaining knowledge about temporal intervals
Communications of the ACM
“Put-that-there”: Voice and gesture at the graphics interface
SIGGRAPH '80 Proceedings of the 7th annual conference on Computer graphics and interactive techniques
Toward a theory of organized multimodal integration patterns during human-computer interaction
Proceedings of the 5th international conference on Multimodal interfaces
ICARE: a component-based approach for the design and development of multimodal interfaces
CHI '04 Extended Abstracts on Human Factors in Computing Systems
ICARE software components for rapidly developing multimodal interfaces
Proceedings of the 6th international conference on Multimodal interfaces
Context based multimodal fusion
Proceedings of the 6th international conference on Multimodal interfaces
Individual differences in multimodal integration patterns: what are they and why do they exist?
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
SketchWizard: Wizard of Oz prototyping of pen-based user interfaces
Proceedings of the 20th annual ACM symposium on User interface software and technology
An integrative recognition method for speech and gestures
ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
A framework for the combination and characterization of output modalities
DSV-IS'00 Proceedings of the 7th international conference on Design, specification, and verification of interactive systems
Service-oriented autonomic multimodal interaction in a pervasive environment
ICMI '11 Proceedings of the 13th international conference on multimodal interfaces
Hi-index | 0.00 |
The CARE properties (Complementarity, Assignment, Redundancy and Equivalence) define various forms that multimodal input interaction can take. While Equivalence and Assignment express the availability and respective absence of choice between multiple input modalities for performing a given task, Complementarity and Redundancy describe relationships between modalities and require fusion mechanisms. In this paper we present a summary of the works we have carried using the CARE properties for conceiving and implementing multimodal interaction, as well as a new approach using WoZ components. We present different technical solutions for implementing the Complementarity and Redundancy of modalities with a focus on the temporal aspects of the fusion. Starting from a monolithic fusion mechanism, we then explain our component-based approach and the composition components (i.e., Redundancy and Complementarity components). As a new contribution for exploring design solutions before implementing an adequate fusion mechanism as well as for tuning the temporal aspects of the performed fusion, we introduce Wizard of Oz (WoZ) fusion components. We illustrate the composition components as well as the implemented tools exploiting them using several multimodal systems including a multimodal slide viewer and a multimodal map navigator.