AVI '94 Proceedings of the workshop on Advanced visual interfaces
A transformational approach for multimodal web user interfaces based on UsiXML
ICMI '05 Proceedings of the 7th international conference on Multimodal interfaces
COMET(s), A Software Architecture Style and an Interactors Toolkit for Plastic User Interfaces
Interactive Systems. Design, Specification, and Verification
Executable Models for Human-Computer Interaction
Interactive Systems. Design, Specification, and Verification
Proceedings of the 1st ACM SIGCHI symposium on Engineering interactive computing systems
The WebTaskModel approach to web process modelling
TAMODIA'07 Proceedings of the 6th international conference on Task models and diagrams for user interface design
USIXML: a language supporting multi-path development of user interfaces
EHCI-DSVIS'04 Proceedings of the 2004 international conference on Engineering Human Computer Interaction and Interactive Systems
Model-based design and generation of a gesture-based user interface navigation control
Proceedings of the 10th Brazilian Symposium on on Human Factors in Computing Systems and the 5th Latin American Conference on Human-Computer Interaction
Engineering device-spanning, multimodal web applications using a model-based design approach
Proceedings of the 18th Brazilian symposium on Multimedia and the web
| Hi-index | 0.00 |
Future interaction will be embedded into smart environments offering the user to choose and to combine a heterogeneous set of interaction devices and modalities based on his preferences realizing an ubiquitous and multimodal access. We propose a model-based runtime environment (the MINT Framework) that describes multimodal interaction by interactors and multimodal mappings. The interactors are modeled by using state machines and describe user interface elements for various modalities. Mappings combine these interactors with interaction devices and support the definition of multimodal relations. We prove our implementation by modeling a multimodal navigation supporting pointing and hand gestures. We additionally present the flexibility of our approach that supports modeling of common interaction paradigms such as drag-and-drop as well.