Design patterns: elements of reusable object-oriented software
Design patterns: elements of reusable object-oriented software
A generic platform for addressing the multimodal challenge
CHI '95 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
A framework and toolkit for the construction of multimodal learning interfaces
A framework and toolkit for the construction of multimodal learning interfaces
Unification-based multimodal integration
ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
A conceptual framework for developing adaptive multimodal applications
Proceedings of the 11th international conference on Intelligent user interfaces
A speech mashup framework for multimodal mobile services
Proceedings of the 2009 international conference on Multimodal interfaces
| Hi-index | 0.00 |
In this paper we present a solution to compose synchronized multimodal computer human interaction with a pattern-oriented approach. The focus of this paper is how to synchronize the different modalities of interaction.In our application scenario the system consists of mobile agents and the user interface is generated from a dialog description language, which is XML-encoded. The generated User Interface is executed by one or more UI-engines, which can use one or more modalities like a text chat system, a speech-based system or a graphical, window-oriented user interface. The user interface is separated and also structured by so-called dialog moves which are suitable to be adapted to different UI-engines. The system supports more than one UI-engine at the same time which results in a possibly multimodal interaction. The interaction is synchronized by observing the events produced by the dialog moves of each UI-engine.The system is evaluated in a project called MAP1 which deals with new human computer interaction methods in future mobile work environments.