BT Technology Journal
Connecting language to the world
Artificial Intelligence - Special volume on connecting language to the world
Semiotic schemas: a framework for grounding language in action and perception
Artificial Intelligence - Special volume on connecting language to the world
Ontology-based speech act identification in a bilingual dialog system using partial pattern trees
Journal of the American Society for Information Science and Technology
Object schemas for responsive robotic language use
Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction
Beyond the individual: new insights on language, cognition and robots
Connection Science - Language and Robots
Object schemas for grounding language in a responsive robot
Connection Science - Language and Robots
Connecting language to the world
Artificial Intelligence - Special volume on connecting language to the world
Semiotic schemas: A framework for grounding language in action and perception
Artificial Intelligence - Special volume on connecting language to the world
Intentional context in situated natural language learning
CONLL '05 Proceedings of the Ninth Conference on Computational Natural Language Learning
Visual query expansion via incremental hypernetwork models of image and text
PRICAI'10 Proceedings of the 11th Pacific Rim international conference on Trends in artificial intelligence
Lingodroids: socially grounding place names in privately grounded cognitive maps
Adaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems
Engaging in a conversation with synthetic characters along the virtuality continuum
SG'05 Proceedings of the 5th international conference on Smart Graphics
Beyond here-and-now: extending shared physical experiences to shared conceptual experiences
Adaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems
Sketch-editing games: human-machine communication, game theory and applications
Proceedings of the 25th annual ACM symposium on User interface software and technology
Integrating word acquisition and referential grounding towards physical world interaction
Proceedings of the 14th ACM international conference on Multimodal interaction
SAFECOMP'12 Proceedings of the 2012 international conference on Computer Safety, Reliability, and Security
Special issue on artificial mental imagery in cognitive systems and robotics
Adaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems
Hi-index | 0.00 |
To build robots that engage in fluid face-to-face spoken conversations with people, robots must have ways to connect what they say to what they see. A critical aspect of how language connects to vision is that language encodes points of view. The meaning of my left and your left differs due to an implied shift of visual perspective. The connection of language to vision also relies on object permanence. We can talk about things that are not in view. For a robot to participate in situated spoken dialog, it must have the capacity to imagine shifts of perspective, and it must maintain object permanence. We present a set of representations and procedures that enable a robotic manipulator to maintain a "mental model" of its physical environment by coupling active vision to physical simulation. Within this model, "imagined" views can be generated from arbitrary perspectives, providing the basis for situated language comprehension and production. An initial application of mental imagery for spatial language understanding for an interactive robot is described.