Synergistic use of direct manipulation and natural language
CHI '89 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
QuickSet: multimodal interaction for distributed applications
MULTIMEDIA '97 Proceedings of the fifth ACM international conference on Multimedia
MVIEWS: multimodal tools for the video analyst
IUI '98 Proceedings of the 3rd international conference on Intelligent user interfaces
Integrating Faces and Fingerprints for Personal Identification
IEEE Transactions on Pattern Analysis and Machine Intelligence
Mutual disambiguation of recognition errors in a multimodel architecture
Proceedings of the SIGCHI conference on Human Factors in Computing Systems
Taming recognition errors with a multimodal interface
Communications of the ACM
Multimodal system processing in mobile environments
UIST '00 Proceedings of the 13th annual ACM symposium on User interface software and technology
Handbook of Multimodal and Spoken Dialogue Systems: Resources, Terminology and Product Evaluation
Handbook of Multimodal and Spoken Dialogue Systems: Resources, Terminology and Product Evaluation
Person Identification Using Multiple Cues
IEEE Transactions on Pattern Analysis and Machine Intelligence
Combining Evidence in Multimodal Personal Identity Recognition Systems
AVBPA '97 Proceedings of the First International Conference on Audio- and Video-Based Biometric Person Authentication
Information Fusion in Biometrics
AVBPA '01 Proceedings of the Third International Conference on Audio- and Video-Based Biometric Person Authentication
The human-computer interaction handbook
Multimodal Interfaces for Multimedia Information Agents
ICASSP '97 Proceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '97) -Volume 1 - Volume 1
Automatic lipreading to enhance speech recognition (speech reading)
Automatic lipreading to enhance speech recognition (speech reading)
Integrating audio and visual information to provide highly robust speech recognition
ICASSP '96 Proceedings of the Acoustics, Speech, and Signal Processing, 1996. on Conference Proceedings., 1996 IEEE International Conference - Volume 02
Multimodal interactive maps: designing for human performance
Human-Computer Interaction
Multimodal integration-a statistical view
IEEE Transactions on Multimedia
Audio-visual speech modeling for continuous speech recognition
IEEE Transactions on Multimedia
Biological and cognitive foundations of intelligent sensor fusion
IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans
Multimodal Interfaces: A Survey of Principles, Models and Frameworks
Human Machine Interaction
Visual search-based design and evaluation of screen magnifiers for older and visually impaired users
International Journal of Human-Computer Studies
Error Rate in Multimodal Mobility Systems
Proceedings of the 2006 conference on Advances in Intelligent IT: Active Media Technology 2006
Intelligent Interface for Elderly Games
UAHCI '09 Proceedings of the 5th International on ConferenceUniversal Access in Human-Computer Interaction. Part II: Intelligent and Ubiquitous Interaction Environments
A dynamic Bayesian approach to computational Laban shape quality analysis
Advances in Human-Computer Interaction
Designing of multimodal feedback for enhanced multitasking performance
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Self-adapting TV based applications
UAHCI'11 Proceedings of the 6th international conference on Universal access in human-computer interaction: design for all and eInclusion - Volume Part I
Mudra: a unified multimodal interaction framework
ICMI '11 Proceedings of the 13th international conference on multimodal interfaces
Service-oriented autonomic multimodal interaction in a pervasive environment
ICMI '11 Proceedings of the 13th international conference on multimodal interfaces
Modeling multimodal integration with event logic charts
Proceedings of the 14th ACM international conference on Multimodal interaction
We need to talk: HCI and the delicate topic of spoken language interaction
CHI '13 Extended Abstracts on Human Factors in Computing Systems
i*Chameleon: a platform for developing multimodal application with comprehensive development cycle
Proceedings of the 28th Annual ACM Symposium on Applied Computing
A dialogue system for multimodal human-robot interaction
Proceedings of the 15th ACM on International conference on multimodal interaction
Review Article: Multimodal interaction: A review
Pattern Recognition Letters
| Hi-index | 0.00 |
A well-designed multimodal interface that fuses two or more information sources can be an effective means of substantially reducing recognition uncertainty. Robustness advantages in multimodal systems have been demonstrated for different modality combinations, for varied tasks, and in different usage environments. Perhaps most importantly, the error suppression achievable with a multimodal system, compared with a unimodal one, can be in excess of 40 percent. In addition to improving overall recognition rates, a multimodal interface can perform in a more stable and effective manner across a variety of challenging user groups and real-world settings. This article reviews recent demonstrations of enhanced robustness for three types of multimodal interfaces, including ones that process speech and pen, speech and lip movement, and multibiometrics (physiological and behavioral) inputs. It concludes by discussing general design strategies for optimizing the robustness of future multimodal systems.