Natural language understanding (2nd ed.)
Natural language understanding (2nd ed.)
A computational model of color perception and color naming
A computational model of color perception and color naming
Normalized Cuts and Image Segmentation
IEEE Transactions on Pattern Analysis and Machine Intelligence
International Journal of Human-Computer Studies - Special issue on collaboration, cooperation and conflict in dialogue systems
Design Considerations for Generic Grouping in Vision
IEEE Transactions on Pattern Analysis and Machine Intelligence
A Grouping Principle and Four Applications
IEEE Transactions on Pattern Analysis and Machine Intelligence
Karma: knowledge-based active representations for metaphor and aspect
Karma: knowledge-based active representations for metaphor and aspect
When push comes to shove: a computational model of the role of motor control in the acquisition of action verbs
Augmenting user interfaces with adaptive speech commands
Proceedings of the 5th international conference on Multimodal interfaces
ACL '03 Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - Volume 1
Grounding the lexical semantics of verbs in visual perception using force dynamics and event logic
Journal of Artificial Intelligence Research
Ubiquitous talker: spoken language interaction with real world objects
IJCAI'95 Proceedings of the 14th international joint conference on Artificial intelligence - Volume 2
Exploiting a sensed environment to improve human-agent communication
Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems
Probabilistic grounding of situated speech using plan recognition and reference resolution
ICMI '05 Proceedings of the 7th international conference on Multimodal interfaces
Proceedings of the 2005 joint conference on Smart objects and ambient intelligence: innovative context-aware services: usages and technologies
Connecting language to the world
Artificial Intelligence - Special volume on connecting language to the world
Learning to talk about events from narrated video in a construction grammar framework
Artificial Intelligence - Special volume on connecting language to the world
Artificial Intelligence - Special volume on connecting language to the world
Choosing words in computer-generated weather forecasts
Artificial Intelligence - Special volume on connecting language to the world
Semiotic schemas: a framework for grounding language in action and perception
Artificial Intelligence - Special volume on connecting language to the world
Conversational robots: building blocks for grounding word meaning
HLT-NAACL-LWM '04 Proceedings of the HLT-NAACL 2003 workshop on Learning word meaning from non-linguistic data - Volume 6
ACL-44 Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics
Incremental generation of spatial referring expressions in situated dialog
ACL-44 Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics
Interpreting vague utterances in context
COLING '04 Proceedings of the 20th international conference on Computational Linguistics
Elements of a spoken language programming interface for robots
Proceedings of the ACM/IEEE international conference on Human-robot interaction
Attention driven reference resolution in multimodal contexts
Artificial Intelligence Review
A computational model for spatial expression resolution
Proceedings of the 9th international conference on Multimodal interfaces
Exploiting referential context in spoken language interfaces for data-poor domains
Proceedings of the 13th international conference on Intelligent user interfaces
Salience-driven Contextual Priming of Speech Recognition for Human-Robot Interaction
Proceedings of the 2008 conference on ECAI 2008: 18th European Conference on Artificial Intelligence
Applying computational models of spatial prepositions to visually situated dialog
Computational Linguistics
Societal grounding is essential to meaningful language use
AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1
EACL '06 Proceedings of the Eleventh Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop
HLT-NAACL-Short '04 Proceedings of HLT-NAACL 2004: Short Papers
A framework for fast incremental interpretation during speech decoding
Computational Linguistics
Semantic scene concept learning by an autonomous agent
AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 2
A simple method for resolution of definite reference in a shared visual context
SIGdial '08 Proceedings of the 9th SIGdial Workshop on Discourse and Dialogue
Cognitive principles in robust multimodal interpretation
Journal of Artificial Intelligence Research
Connecting language to the world
Artificial Intelligence - Special volume on connecting language to the world
Learning to talk about events from narrated video in a construction grammar framework
Artificial Intelligence - Special volume on connecting language to the world
Semiotic schemas: A framework for grounding language in action and perception
Artificial Intelligence - Special volume on connecting language to the world
Artificial Intelligence - Special volume on connecting language to the world
Choosing words in computer-generated weather forecasts
Artificial Intelligence - Special volume on connecting language to the world
Discourse factors influencing spatial descriptions in English and German
SC'06 Proceedings of the 2006 international conference on Spatial Cognition V: reasoning, action, interaction
Context-based word acquisition for situated dialogue in a virtual world
Journal of Artificial Intelligence Research
A game-theoretic approach to generating spatial descriptions
EMNLP '10 Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing
Proceedings of the 2010 workshop on Eye gaze in intelligent human machine interaction
A model of spatial reference frames in language
COSIT'11 Proceedings of the 10th international conference on Spatial information theory
Intelligent management of data driven simulations to support model building in the social sciences
ICCS'06 Proceedings of the 6th international conference on Computational Science - Volume Part III
Computational generation of referring expressions: A survey
Computational Linguistics
Integrating word acquisition and referential grounding towards physical world interaction
Proceedings of the 14th ACM international conference on Multimodal interaction
Towards mediating shared perceptual basis in situated dialogue
SIGDIAL '12 Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Hi-index | 0.00 |
We present a visually-grounded language understanding model based on a study of how people verbally describe objects in scenes. The emphasis of the model is on the combination of individual word meanings to produce meanings for complex referring expressions. The model has been implemented, and it is able to understand a broad range of spatial referring expressions. We describe our implementation of word level visually-grounded semantics and their embedding in a compositional parsing framework. The implemented system selects the correct referents in response to natural language expressions for a large percentage of test cases. In an analysis of the system's successes and failures we reveal how visual context influences the semantics of utterances and propose future extensions to the model that take such context into account.