Integration and synchronization of input modes during multimodal human-computer interaction
Proceedings of the ACM SIGCHI Conference on Human factors in computing systems
Ten myths of multimodal interaction
Communications of the ACM
Using critical path analysis to model multimodal human-computer interaction
International Journal of Human-Computer Studies
A randomized algorithm for finding a path subject to multiple QoS requirements
Computer Networks: The International Journal of Computer and Telecommunications Networking
Computers and Intractability; A Guide to the Theory of NP-Completeness
Computers and Intractability; A Guide to the Theory of NP-Completeness
Introducing the Adaptive-QoS Idea into Multi-Tier UMTS Systems: The Lessons Learned
Wireless Personal Communications: An International Journal
Advanced Interaction in Context
HUC '99 Proceedings of the 1st international symposium on Handheld and Ubiquitous Computing
SenSay: A Context-Aware Mobile Phone
ISWC '03 Proceedings of the 7th IEEE International Symposium on Wearable Computers
Understanding contexts by being there: case studies in bodystorming
Personal and Ubiquitous Computing
A framework for rapid development of multimodal interfaces
Proceedings of the 5th international conference on Multimodal interfaces
A study of digital ink in lecture presentation
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
The Input Configurator toolkit: towards high input adaptability in interactive applications
Proceedings of the working conference on Advanced visual interfaces
Dynamic Configuration of Resource-Aware Services
Proceedings of the 26th International Conference on Software Engineering
Speech, ink, and slides: the interaction of content channels
Proceedings of the 12th annual ACM international conference on Multimedia
ICARE software components for rapidly developing multimodal interfaces
Proceedings of the 6th international conference on Multimodal interfaces
Multimodal output specification / simulation platform
ICMI '05 Proceedings of the 7th international conference on Multimodal interfaces
A conceptual framework for developing adaptive multimodal applications
Proceedings of the 11th international conference on Intelligent user interfaces
A versatile large-scale multimodal VR system for cultural heritage visualization
Proceedings of the ACM symposium on Virtual reality software and technology
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Universal accessibility as a multimodal design issue
Communications of the ACM - ACM at sixty: a look back in time
Finding a path subject to many additive QoS constraints
IEEE/ACM Transactions on Networking (TON)
Defining scenarios for mobile design and evaluation
CHI '08 Extended Abstracts on Human Factors in Computing Systems
CARISMA: Context-Aware Reflective mIddleware System for Mobile Applications
IEEE Transactions on Software Engineering
Context-Aware Computing Applications
WMCSA '94 Proceedings of the 1994 First Workshop on Mobile Computing Systems and Applications
Designing and evaluating multimodal interaction for mobile contexts
ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
Tech-note: Multimodal feedback in 3D target acquisition
3DUI '09 Proceedings of the 2009 IEEE Symposium on 3D User Interfaces
Decision-theoretic user interface generation
AAAI'08 Proceedings of the 23rd national conference on Artificial intelligence - Volume 3
Multi-modal exploration of small artifacts: an exhibition at the Gold Museum in Bogota
Proceedings of the 16th ACM Symposium on Virtual Reality Software and Technology
Multimodal human computer interaction: a survey
ICCV'05 Proceedings of the 2005 international conference on Computer Vision in Human-Computer Interaction
Gumo: the general user model ontology
UM'05 Proceedings of the 10th international conference on User Modeling
Cognitive Resource-Aware Adaptive Web Service Binding and Scheduling
WI-IAT '12 Proceedings of the The 2012 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology - Volume 01
Design guidelines for adaptive multimodal mobile input solutions
Proceedings of the 15th international conference on Human-computer interaction with mobile devices and services
Hi-index | 0.00 |
Multimodal interfaces have attracted more and more attention. Most researches focus on each interaction mode independently and then fuse information at the application level. Recently, several frameworks and models have been proposed to support the design and development of multimodal interfaces. However, it is challenging to provide automatic modality adaptation in multimodal interfaces. Existing approaches are using rule-based specifications to define the adaptation of input/output modalities. Rule-based specifications have the problems of completeness and coherence. Distinct from previous work, this paper presents a novel approach that quantifies the user preference of each modality and considers the adaptation as an optimization issue that searches for a set of input/output modalities matching user's preference. Our approach applies a cross-layer design, which considers the adaptation from the perspectives of the interaction context, available system resources, and QoS requirements. Furthermore, our approach supports human-centric adaptation. A user can report the preference of a modality so that selected modalities fit user's personal needs. An optimal solution and a heuristic algorithm have been developed to automatically select an appropriate set of modality combinations under a specific situation. We have designed a framework based on the heuristic algorithm and existing ontology, and applied the framework to conduct a utility evaluation, in which we have employed a within-subject experiment. Fifty participants were invited to go through three scenarios and compare automatically selected modalities with randomly selected modalities. The results from the experiment show that users perceived the automatically selected modalities as appropriate and satisfactory.