Readings in natural language processing
So what can we talk about now?
Readings in natural language processing
Synergistic use of direct manipulation and natural language
CHI '89 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Knowledge-based simulation at the RAND Corporation
Knowledge-based simulation
The perspective wall: detail and context smoothly integrated
CHI '91 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Intelligent multi-media interface technology
Intelligent user interfaces
User and discourse models for multimodal communication
Intelligent user interfaces
An intelligent interface architecture for adaptive interaction
Intelligent user interfaces
Can Drawing Be Liberated From the Von Neumann Style?
Databases for Business and Office Applications, Database Week
The pragmatics of referring and the modality of communication
Computational Linguistics
An efficient easily adaptable system for interpreting natural language queries
Computational Linguistics
Coping with syntactic ambiguity or how to put the block in the box on the table
Computational Linguistics
Providing a unified account of definite noun phrases in discourse
ACL '83 Proceedings of the 21st annual meeting on Association for Computational Linguistics
ACL '80 Proceedings of the 18th annual meeting on Association for Computational Linguistics
The interpretation of tense and aspect in English
ACL '88 Proceedings of the 26th annual meeting on Association for Computational Linguistics
The future of speech and audio in the interface: a CHI '94 workshop
ACM SIGCHI Bulletin
An improved interface for tutorial dialogues: browsing a visual dialogue history
CHI '94 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Automatic referent resolution of deictic and anaphoric expressions
Computational Linguistics
Adding a collaborative agent to graphical user interfaces
Proceedings of the 9th annual ACM symposium on User interface software and technology
QuickSet: multimodal interaction for distributed applications
MULTIMEDIA '97 Proceedings of the fifth ACM international conference on Multimedia
COLLAGEN: when agents collaborate with people
AGENTS '97 Proceedings of the first international conference on Autonomous agents
Task integration in multimodal speech recognition environments
Crossroads - Special issue on human computer interaction
Design principles for intelligent environments
AAAI '98/IAAI '98 Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence
The integrality of speech in multimodal interfaces
ACM Transactions on Computer-Human Interaction (TOCHI)
Put that where? voice and gesture at the graphics interface
ACM SIGGRAPH Computer Graphics
Supporting creative work tasks: the potential of multimodal tools to support sketching
C&C '99 Proceedings of the 3rd conference on Creativity & cognition
Towards a computational model of sketching
Proceedings of the 6th international conference on Intelligent user interfaces
COLLAGEN: A Collaboration Manager for Software Interface Agents
User Modeling and User-Adapted Interaction
Natural Language Access to Public Administration Data: The TAMIC-P System
AI*IA '99 Proceedings of the 6th Congress of the Italian Association for Artificial Intelligence on Advances in Artificial Intelligence
How synthetic characters can help decision making
Decision making support systems
Adding intelligence to the interface
VRAIS '96 Proceedings of the 1996 Virtual Reality Annual International Symposium (VRAIS 96)
Multimodal Dialogue Systems for Interactive TVApplications
ICMI '02 Proceedings of the 4th IEEE International Conference on Multimodal Interfaces
The role of spoken feedback in experiencing multimodal interfaces as human-like
Proceedings of the 5th international conference on Multimodal interfaces
QuickSet: multimodal interaction for simulation set-up and control
ANLC '97 Proceedings of the fifth conference on Applied natural language processing
Unification-based multimodal integration
ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
Multisensory interaction metaphors with haptics and proprioception in virtual environments
Proceedings of the third Nordic conference on Human-computer interaction
Multimodal interaction with xforms
ICWE '06 Proceedings of the 6th international conference on Web engineering
Exploiting proprioception to improve haptic interaction in a virtual environment
Presence: Teleoperators and Virtual Environments - Special issue: 2004 workshop on VR design and evaluation
HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces
HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces
Tlk or txt? Using voice input for SMS composition
Personal and Ubiquitous Computing
Development of a dual-modal information presentation of sequential relationship
International Journal of Mobile Learning and Organisation
Robust understanding in multimodal interfaces
Computational Linguistics
Multimodal interaction: a new focal area for AI
IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 2
Manual deskterity: an exploration of simultaneous pen + touch direct input
CHI '10 Extended Abstracts on Human Factors in Computing Systems
Multimodal dialogue systems: a case study for interactive TV
ERCIM'02 Proceedings of the User interfaces for all 7th international conference on Universal access: theoretical perspectives, practice, and experience
UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology
Mulitmodal interaction for distributed interactive simulation
AAAI'97/IAAI'97 Proceedings of the fourteenth national conference on artificial intelligence and ninth conference on Innovative applications of artificial intelligence
The long-term adoption of speech recognition in medical applications
CBMS'03 Proceedings of the 16th IEEE conference on Computer-based medical systems
An evaluation of an augmented reality multimodal interface using speech and paddle gestures
ICAT'06 Proceedings of the 16th international conference on Advances in Artificial Reality and Tele-Existence
Managing implicit assumptions in natural language interfaces
Proceedings of the 2012 ACM international conference on Intelligent User Interfaces
AmbiLearn: Multimodal Assisted Learning
International Journal of Ambient Computing and Intelligence
Hi-index | 0.00 |
Although graphics and direct manipulation are effective interface technologies for some classes of problems, they are limited in many ways. In particular, they provide little support for identifying objects not on the screen, for specifying temporal relations, for identifying and operating on large sets and subsets of entities, and for using the context of interaction. On the other hand, these are precisely strengths of natural language. This paper presents an interface that blends natural language processing and direct manipulation technologies, using each for their characteristic advantages. Specifically, the paper shows how to use natural language to describe objects and temporal relations, and how to use direct manipulation for overcoming hard natural language problems involving the establishment and use of context and pronominal reference. This work has been implemented in SRI's Shoptalk system, a prototype information and decision-support system for manufacturing.