Text generation: using discourse strategies and focus constraints to generate natural language text
Text generation: using discourse strategies and focus constraints to generate natural language text
Intelligent multi-media interface technology
Intelligent user interfaces
Integrating simultaneous input from speech, gaze, and hand gestures
Intelligent multimedia interfaces
On the knowledge underlying multimedia presentations
Intelligent multimedia interfaces
A design space for multimodal systems: concurrent processing and data fusion
CHI '93 Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems
Participating in explanatory dialogues: interpreting and responding to questions in context
Participating in explanatory dialogues: interpreting and responding to questions in context
QuickSet: multimodal interaction for distributed applications
MULTIMEDIA '97 Proceedings of the fifth ACM international conference on Multimedia
Survey of the state of the art in human language technology
Manual and gaze input cascaded (MAGIC) pointing
Proceedings of the SIGCHI conference on Human Factors in Computing Systems
Ten myths of multimodal interaction
Communications of the ACM
Perceptual user interfaces (introduction)
Communications of the ACM
Phidgets: easy development of physical interfaces through physical widgets
Proceedings of the 14th annual ACM symposium on User interface software and technology
The Psychology of Human-Computer Interaction
The Psychology of Human-Computer Interaction
The human-computer interaction handbook
“Put-that-there”: Voice and gesture at the graphics interface
SIGGRAPH '80 Proceedings of the 7th annual conference on Computer graphics and interactive techniques
A Real-Time Framework for Natural Multimodal Interaction with Large Screen Displays
ICMI '02 Proceedings of the 4th IEEE International Conference on Multimodal Interfaces
Automatic lipreading to enhance speech recognition (speech reading)
Automatic lipreading to enhance speech recognition (speech reading)
Advances in Robust Multimodal Interface Design
IEEE Computer Graphics and Applications
Toward a theory of organized multimodal integration patterns during human-computer interaction
Proceedings of the 5th international conference on Multimodal interfaces
A framework for rapid development of multimodal interfaces
Proceedings of the 5th international conference on Multimodal interfaces
XISL: a language for describing multimodal interaction scenarios
Proceedings of the 5th international conference on Multimodal interfaces
Guidelines for multimodal user interface design
Communications of the ACM - Multimodal interfaces that flex, adapt, and persist
Unification-based multimodal integration
ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
Papier-Mache: toolkit support for tangible input
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
ICARE software components for rapidly developing multimodal interfaces
Proceedings of the 6th international conference on Multimodal interfaces
A transformational approach for multimodal web user interfaces based on UsiXML
ICMI '05 Proceedings of the 7th international conference on Multimodal interfaces
Human-centered design meets cognitive load theory: designing interfaces that help people think
MULTIMEDIA '06 Proceedings of the 14th annual ACM international conference on Multimedia
From vocal to multimodal dialogue management
Proceedings of the 8th international conference on Multimodal interfaces
Archivus: a multimodal system for multimedia meeting browsing and retrieval
COLING-ACL '06 Proceedings of the COLING/ACL on Interactive presentation sessions
CHI '07 Extended Abstracts on Human Factors in Computing Systems
ICASSP '96 Proceedings of the Acoustics, Speech, and Signal Processing, 1996. on Conference Proceedings., 1996 IEEE International Conference - Volume 06
Multimodal human-computer interaction: A survey
Computer Vision and Image Understanding
Proceedings of the 2nd international conference on Tangible and embedded interaction
The openinterface framework: a tool for multimodal interaction.
CHI '08 Extended Abstracts on Human Factors in Computing Systems
Graphical representation of meetings on mobile devices
Proceedings of the 10th international conference on Human computer interaction with mobile devices and services
Multimodal interactive maps: designing for human performance
Human-Computer Interaction
Multimodal dialog description language for rapid system development
SigDIAL '06 Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue
FaericWorld: browsing multimedia events through static documents and links
INTERACT'07 Proceedings of the 11th IFIP TC 13 international conference on Human-computer interaction
An ego-centric and tangible approach to meeting indexing and browsing
MLMI'07 Proceedings of the 4th international conference on Machine learning for multimodal interaction
Multimodal integration-a statistical view
IEEE Transactions on Multimedia
From members to teams to committee-a robust approach to gestural and multimodal recognition
IEEE Transactions on Neural Networks
Evaluating multimodal systems: a comparison of established questionnaires and interaction parameters
Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries
Head-computer interface: a multimodal approach to navigate through real and virtual worlds
HCII'11 Proceedings of the 14th international conference on Human-computer interaction: interaction techniques and environments - Volume Part II
Eliciting interaction requirements for adaptive multimodal TV based applications
UAHCI'11 Proceedings of the 6th international conference on Universal access in human-computer interaction: design for all and eInclusion - Volume Part I
The contribution of multimodal adaptation techniques to the GUIDE interface
UAHCI'11 Proceedings of the 6th international conference on Universal access in human-computer interaction: design for all and eInclusion - Volume Part I
Adapting multimodal fission to user's abilities
UAHCI'11 Proceedings of the 6th international conference on Universal access in human-computer interaction: design for all and eInclusion - Volume Part I
Self-adapting TV based applications
UAHCI'11 Proceedings of the 6th international conference on Universal access in human-computer interaction: design for all and eInclusion - Volume Part I
UAHCI'11 Proceedings of the 6th international conference on Universal access in human-computer interaction: design for all and eInclusion - Volume Part I
ARAMIS: toward a hybrid approach for human- environment interaction
HCII'11 Proceedings of the 14th international conference on Human-computer interaction: towards mobile and intelligent interaction environments - Volume Part III
Towards a semantic modelling framework in support of multimodal user interface design
INTERACT'11 Proceedings of the 13th IFIP TC 13 international conference on Human-computer interaction - Volume Part IV
Monte carlo methods for managing interactive state, action and feedback under uncertainty
Proceedings of the 24th annual ACM symposium on User interface software and technology
Mudra: a unified multimodal interaction framework
ICMI '11 Proceedings of the 13th international conference on multimodal interfaces
Emerging Input Technologies for Always-Available Mobile Interaction
Foundations and Trends in Human-Computer Interaction
Engineering device-spanning, multimodal web applications using a model-based design approach
Proceedings of the 18th Brazilian symposium on Multimedia and the web
KI'12 Proceedings of the 35th Annual German conference on Advances in Artificial Intelligence
Adaptive probabilistic fission for multimodal systems
Proceedings of the 24th Australian Computer-Human Interaction Conference
UCAmI'12 Proceedings of the 6th international conference on Ubiquitous Computing and Ambient Intelligence
Cognitive Resource-Aware Adaptive Web Service Binding and Scheduling
WI-IAT '12 Proceedings of the The 2012 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology - Volume 01
Using the transferable belief model for multimodal input fusion in companion systems
MPRSS'12 Proceedings of the First international conference on Multimodal Pattern Recognition of Social Signals in Human-Computer-Interaction
Assessing the support provided by a toolkit for rapid prototyping of multimodal systems
Proceedings of the 5th ACM SIGCHI symposium on Engineering interactive computing systems
The CoGenIVE concept revisited: a toolkit for prototyping multimodal systems
Proceedings of the 5th ACM SIGCHI symposium on Engineering interactive computing systems
Design guidelines for adaptive multimodal mobile input solutions
Proceedings of the 15th international conference on Human-computer interaction with mobile devices and services
Multimodal eyes-free exploration of maps: TIKISI for maps
ACM SIGACCESS Accessibility and Computing
A dialogue system for multimodal human-robot interaction
Proceedings of the 15th ACM on International conference on multimodal interaction
Comparing the multimodal interaction technique design of MINT with NiMMiT
Proceedings of the 19th Brazilian symposium on Multimedia and the web
A multimedia presentation system using a 3D gesture interface in museums
Multimedia Tools and Applications
| Hi-index | 0.00 |
The grand challenge of multimodal interface creation is to build reliable processing systems able to analyze and understand multiple communication means in real-time. This opens a number of associated issues covered by this chapter, such as heterogeneous data types fusion, architectures for real-time processing, dialog management, machine learning for multimodal interaction, modeling languages, frameworks, etc. This chapter does not intend to cover exhaustively all the issues related to multimodal interfaces creation and some hot topics, such as error handling, have been left aside. The chapter starts with the features and advantages associated with multimodal interaction, with a focus on particular findings and guidelines, as well as cognitive foundations underlying multimodal interaction. The chapter then focuses on the driving theoretical principles, time-sensitive software architectures and multimodal fusion and fission issues. Modeling of multimodal interaction as well as tools allowing rapid creation of multimodal interfaces are then presented. The article concludes with an outline of the current state of multimodal interaction research in Switzerland, and also summarizes the major future challenges in the field.