Using a knowledge base to drive an expert system interface with a natural language component
Expert systems: the user interface
Multimodal response planning: an adaptive rule based approach
CHI '88 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Can Drawing Be Liberated From the Von Neumann Style?
Databases for Business and Office Applications, Database Week
Towards a Computational Theory of Definite Anaphora Comprehension in English Discourse
Towards a Computational Theory of Definite Anaphora Comprehension in English Discourse
An efficient easily adaptable system for interpreting natural language queries
Computational Linguistics
Coping with syntactic ambiguity or how to put the block in the box on the table
Computational Linguistics
A centering approach to pronouns
ACL '87 Proceedings of the 25th annual meeting on Association for Computational Linguistics
Providing a unified account of definite noun phrases in discourse
ACL '83 Proceedings of the 21st annual meeting on Association for Computational Linguistics
Menu-based natural language understanding
ACL '83 Proceedings of the 21st annual meeting on Association for Computational Linguistics
The interpretation of tense and aspect in English
ACL '88 Proceedings of the 26th annual meeting on Association for Computational Linguistics
Two-handed gesture in multi-modal natural dialog
UIST '92 Proceedings of the 5th annual ACM symposium on User interface software and technology
The role of natural language in a multimodal interface
UIST '92 Proceedings of the 5th annual ACM symposium on User interface software and technology
A survey of design issues in spatial input
UIST '94 Proceedings of the 7th annual ACM symposium on User interface software and technology
HyperActive: extending an open hypermedia architecture to support agency
ACM Transactions on Computer-Human Interaction (TOCHI)
KidSim: end user programming of simulations
CHI '95 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Human-computer interaction through natural language and hypermedia in AlFresco
ACM SIGCHI Bulletin
Segmented interaction history in a collaborative interface agent
Proceedings of the 2nd international conference on Intelligent user interfaces
Multimodal interfaces for dynamic interactive maps
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Integration and synchronization of input modes during multimodal human-computer interaction
Proceedings of the ACM SIGCHI Conference on Human factors in computing systems
A construction tool for context-sensitive guidance system
CHI '93 INTERACT '93 and CHI '93 Conference Companion on Human Factors in Computing Systems
QuickSet: multimodal interaction for distributed applications
MULTIMEDIA '97 Proceedings of the fifth ACM international conference on Multimedia
Mutual disambiguation of recognition errors in a multimodel architecture
Proceedings of the SIGCHI conference on Human Factors in Computing Systems
Integrated interfaces for decision-support with simulation
WSC '91 Proceedings of the 23rd conference on Winter simulation
Towards a computational model of sketching
Proceedings of the 6th international conference on Intelligent user interfaces
Multimodal human discourse: gesture and speech
ACM Transactions on Computer-Human Interaction (TOCHI)
Natural Language in Multimodal Human-Computer Interface
IEEE Expert: Intelligent Systems and Their Applications
Natural Language Access to Public Administration Data: The TAMIC-P System
AI*IA '99 Proceedings of the 6th Congress of the Italian Association for Artificial Intelligence on Advances in Artificial Intelligence
Toward Natural Gesture/Speech Control of a Large Display
EHCI '01 Proceedings of the 8th IFIP International Conference on Engineering for Human-Computer Interaction
The human-computer interaction handbook
Adding intelligence to the interface
VRAIS '96 Proceedings of the 1996 Virtual Reality Annual International Symposium (VRAIS 96)
Multimodal Dialogue Systems for Interactive TVApplications
ICMI '02 Proceedings of the 4th IEEE International Conference on Multimodal Interfaces
Advances in Robust Multimodal Interface Design
IEEE Computer Graphics and Applications
Visual display, pointing, and natural language: the power of multimodal interaction
AVI '98 Proceedings of the working conference on Advanced visual interfaces
The efficiency of multimodal interaction for a map-based task
ANLC '00 Proceedings of the sixth conference on Applied natural language processing
Integrating natural language components into graphical discourse
ANLC '92 Proceedings of the third conference on Applied natural language processing
QuickSet: multimodal interaction for simulation set-up and control
ANLC '97 Proceedings of the fifth conference on Applied natural language processing
Review of "Intelligent multimedia interfaces" by Mark T. Maybury. AAAI Press and The MIT Press 1993.
Computational Linguistics - Special issue on computational phonology
The representation of multimodal user interface dialogues using discourse pegs
ACL '92 Proceedings of the 30th annual meeting on Association for Computational Linguistics
Drawing pictures with natural language and direct manipulation
COLING '94 Proceedings of the 15th conference on Computational linguistics - Volume 2
Multi-Modal Definite Clause Grammar
COLING '94 Proceedings of the 15th conference on Computational linguistics - Volume 2
COLING '92 Proceedings of the 14th conference on Computational linguistics - Volume 4
Multi-Modal-Method: a design method for building multi-modal systems
COLING '96 Proceedings of the 16th conference on Computational linguistics - Volume 2
Elvis: situated speech and gesture understanding for a robotic chandelier
Proceedings of the 6th international conference on Multimodal interfaces
Linking messages and form requests
Proceedings of the 11th international conference on Intelligent user interfaces
Learning information intent via observation
Proceedings of the 16th international conference on World Wide Web
The catchment feature model: a device for multimodal fusion and a bridge between signal and sense
EURASIP Journal on Applied Signal Processing
HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces
HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces
References to graphical objects in interactive multimodal queries
Knowledge-Based Systems
A Wizard of Oz study for an AR multimodal interface
ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
Multimodal interactive maps: designing for human performance
Human-Computer Interaction
Integration and synchronization of input modes during multimodal human-computer interaction
ReferringPhenomena '97 Referring Phenomena in a Multimedia Context and their Computational Treatment
Natural language and exploration of an information space: the ALFresco interactive system
IJCAI'91 Proceedings of the 12th international joint conference on Artificial intelligence - Volume 2
Multimodal interaction: a new focal area for AI
IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 2
Effects of multimodal feedback on the performance of older adults with normal and impaired vision
ERCIM'02 Proceedings of the User interfaces for all 7th international conference on Universal access: theoretical perspectives, practice, and experience
Sensor synaesthesia: touch in motion, and motion in touch
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
A knowledge elicitation study for a speech enabled GIS to handle vagueness in communication
HCII'11 Proceedings of the 14th international conference on Human-computer interaction: towards mobile and intelligent interaction environments - Volume Part III
An evaluation of an augmented reality multimodal interface using speech and paddle gestures
ICAT'06 Proceedings of the 16th international conference on Advances in Artificial Reality and Tele-Existence
Toward compound navigation tasks on mobiles via spatial manipulation
Proceedings of the 15th international conference on Human-computer interaction with mobile devices and services
HCI'13 Proceedings of the 15th international conference on Human-Computer Interaction: interaction modalities and techniques - Volume Part IV
Hands and speech in space: multimodal interaction with augmented reality interfaces
Proceedings of the 15th ACM on International conference on multimodal interaction
| Hi-index | 0.00 |
This paper shows how the integration of natural language with direct manipulation produces a multimodal interface that overcomes limitations of these techniques when used separately. Natural language helps direct manipulation in being able to specify objects and actions by description, while direct manipulation enables users to learn which objects and actions are available in the system. Furthermore, graphical rendering and manipulation of context provides a partial solution to difficult problems of natural language anaphora.