The human factors of computer graphics interaction techniques
IEEE Computer Graphics and Applications
ACM Transactions on Graphics (TOG)
The NeXT book
Speech and gestures for graphic image manipulation
CHI '89 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Computer graphics: principles and practice (2nd ed.)
Computer graphics: principles and practice (2nd ed.)
Expressive richness: a comparison of speech and text as media for revision
CHI '91 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Dialogue structures for virtual worlds
CHI '91 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Spoken language interfaces: the OM system
CHI '91 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
CHI'90 workshop on multimedia and multimodal interface design
ACM SIGCHI Bulletin
A morphological analysis of the design space of input devices
ACM Transactions on Information Systems (TOIS) - Special issue on computer—human interaction
Proceedings of the IFIP TC2/WG2.7 Working Conference on Engineering for Human-Computer Interaction
Evaluating the usability of user interfaces: Research in practice
INTERACT '90 Proceedings of the IFIP TC13 Third Interational Conference on Human-Computer Interaction
Abstractions for Continuous Media in a Network Window System
Abstractions for Continuous Media in a Network Window System
Lexical and pragmatic considerations of input structures
ACM SIGGRAPH Computer Graphics
Extending a graphical toolkit for two-handed interaction
UIST '94 Proceedings of the 7th annual ACM symposium on User interface software and technology
A generic platform for addressing the multimodal challenge
CHI '95 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Multimodal input for computer access and augmentative communication
Assets '96 Proceedings of the second annual ACM conference on Assistive technologies
Issues and experience in designing two-handed interaction
CHI '94 Conference Companion on Human Factors in Computing Systems
SUITEKeys: a speech understanding interface for the motor-control challenged
Assets '98 Proceedings of the third international ACM conference on Assistive technologies
CHI 98 Cconference Summary on Human Factors in Computing Systems
Interactive environments for music and multimedia
Multimedia Systems - Special issue on audio and multimedia
EGVE '02 Proceedings of the workshop on Virtual environments 2002
EMBASSI: multimodal assistance for universal access to infotainment and service infrastructures
WUAUC'01 Proceedings of the 2001 EC/NSF workshop on Universal accessibility of ubiquitous computing: providing for the elderly
IEEE MultiMedia
Multi-Modal-Method: a design method for building multi-modal systems
COLING '96 Proceedings of the 16th conference on Computational linguistics - Volume 2
Finite-state multimodal integration and understanding
Natural Language Engineering
Steps in Identifying Interaction Design Patterns for Multimodal Systems
HCSE-TAMODIA '08 Proceedings of the 2nd Conference on Human-Centered Software Engineering and 7th International Workshop on Task Models and Diagrams
Names and reference in user interfaces
BCS-HCI '07 Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI...but not as we know it - Volume 2
Multimodal Interfaces: A Survey of Principles, Models and Frameworks
Human Machine Interaction
Benchmarking fusion engines of multimodal interactive systems
Proceedings of the 2009 international conference on Multimodal interfaces
A simple index for multimodal flexibility
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
A framework for the combination and characterization of output modalities
DSV-IS'00 Proceedings of the 7th international conference on Design, specification, and verification of interactive systems
Using visual-textual mutual information and entropy for inter-modal document indexing
ECIR'07 Proceedings of the 29th European conference on IR research
Software architecture for multimodal user input - FLUID
ERCIM'02 Proceedings of the User interfaces for all 7th international conference on Universal access: theoretical perspectives, practice, and experience
Dwell-based pointing in applications of human computer interaction
INTERACT'07 Proceedings of the 11th IFIP TC 13 international conference on Human-computer interaction
Evaluating multimodal systems: a comparison of established questionnaires and interaction parameters
Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries
Head-computer interface: a multimodal approach to navigate through real and virtual worlds
HCII'11 Proceedings of the 14th international conference on Human-computer interaction: interaction techniques and environments - Volume Part II
A pattern-based methodology for multimodal interaction design
TSD'06 Proceedings of the 9th international conference on Text, Speech and Dialogue
Combining modality theory and context models
PIT'06 Proceedings of the 2006 international tutorial and research conference on Perception and Interactive Technologies
Improving BCI performance after classification
Proceedings of the 14th ACM international conference on Multimodal interaction
Assessing the support provided by a toolkit for rapid prototyping of multimodal systems
Proceedings of the 5th ACM SIGCHI symposium on Engineering interactive computing systems
Sharing Stories “in the Wild”: A Mobile Storytelling Case Study Using StoryKit
ACM Transactions on Computer-Human Interaction (TOCHI) - Special Issue of “The Turn to The Wild”
| Hi-index | 0.01 |
Multimodal interaction enables the user to employ different modalities such as voice, gesture and typing for communicating with a computer. This paper presents an analysis of the integration of multiple communication modalities within an interactive system. To do so, a software engineering perspective is adopted. First, the notion of “multimodal system” is clarified. We aim at proving that two main features of a multimodal system are the concurrency of processing and the fusion of input/output data. On the basis of these two features, we then propose a design space and a method for classifying multimodal systems. In the last section, we present a software architecture model of multimodal systems which supports these two salient properties: concurrency of processing and data fusion. Two multimodal systems developed in our team, VoicePaint and NoteBook, are used to illustrate the discussion.