Generating adaptive route instructions using hierarchical reinforcement learning

Authors:
Heriberto Cuayáhuitl;Nina Dethlefs;Lutz Frommberger;Kai-Florian Richter;John Bateman
Affiliations:
Transregional Collaborative Research Center SFB, University of Bremen, Bremen, Germany;Faculty of Linguistics and Literary Sciences, University of Bremen, Bremen, Germany;Transregional Collaborative Research Center SFB, University of Bremen, Bremen, Germany;Transregional Collaborative Research Center SFB, University of Bremen, Bremen, Germany;Transregional Collaborative Research Center SFB, University of Bremen, Bremen, Germany and Faculty of Linguistics and Literary Sciences, University of Bremen, Bremen, Germany
Venue:
SC'10 Proceedings of the 7th international conference on Spatial cognition
Year:
2010

Citing 13
Cited 4

Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
An Overview of MAXQ Hierarchical Reinforcement Learning

SARA '02 Proceedings of the 4th International Symposium on Abstraction, Reformulation, and Approximation
Elements of Good Route Directions in Familiar and Unfamiliar Environments

COSIT '99 Proceedings of the International Conference on Spatial Information Theory: Cognitive and Computational Foundations of Geographic Information Science
The Nature of Landmarks for Real and Electronic Spaces

COSIT '99 Proceedings of the International Conference on Spatial Information Theory: Cognitive and Computational Foundations of Geographic Information Science
Enriching Wayfinding Instructions with Local Landmarks

GIScience '02 Proceedings of the Second International Conference on Geographic Information Science
Spatial Abstraction: Aspectualization, Coarsening, and Conceptual Classification

Proceedings of the international conference on Spatial Cognition VI: Learning, Reasoning, and Talking about Space
Simplest Instructions: Finding Easy-to-Describe Routes for Navigation

GIScience '08 Proceedings of the 5th international conference on Geographic Information Science
Urban granularities--a data structure for cognitively ergonomic route directions

Geoinformatica
Evaluation of a hierarchical reinforcement learning spoken dialogue system

Computer Speech and Language
Hierarchical reinforcement learning with the MAXQ value function decomposition

Journal of Artificial Intelligence Research
Reinforcement learning: a survey

Journal of Artificial Intelligence Research
Algorithms for reliable navigation and wayfinding

SC'06 Proceedings of the 2006 international conference on Spatial Cognition V: reasoning, action, interaction
Context-Specific Route Directions: Generation of Cognitively Motivated Wayfinding Instructions

Context-Specific Route Directions: Generation of Cognitively Motivated Wayfinding Instructions

Spatially-aware dialogue control using hierarchical reinforcement learning

ACM Transactions on Speech and Language Processing (TSLP)
Student profile ergonomically adapted to e-learning. a data clustering and statistical analysis based survey

AIASABEBI'11 Proceedings of the 11th WSEAS international conference on Applied informatics and communications, and Proceedings of the 4th WSEAS International conference on Biomedical electronics and biomedical informatics, and Proceedings of the international conference on Computational engineering in systems applications
Evaluating and minimizing ambiguities in qualitative route instructions

Proceedings of the 19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems
The GRUVE challenge: generating routes under uncertainty in virtual environments

ENLG '11 Proceedings of the 13th European Workshop on Natural Language Generation

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present a learning approach for efficiently inducing adaptive behaviour of route instructions. For such a purpose we propose a two-stage approach to learn a hierarchy of wayfinding strategies using hierarchical reinforcement learning. Whilst the first stage learns low-level behaviour, the second stage focuses on learning high-level behaviour. In our proposed approach, only the latter is to be applied at runtime in user-machine interactions. Our experiments are based on an indoor navigation scenario for a building that is complex to navigate. We compared our approach with flat reinforcement learning and a fully-learnt hierarchical approach. Our experimental results show that our proposed approach learns significantly faster than the baseline approaches. In addition, the learnt behaviour shows to adapt to the type of user and structure of the spatial environment. This approach is attractive to automatic route giving since it combines fast learning with adaptive behaviour.