Accelerating autonomous learning by using heuristic selection of actions

Authors:
Reinaldo A. Bianchi;Carlos H. Ribeiro;Anna H. Costa
Affiliations:
Centro Universitário da FEI, São Bernardo do Campo, Brazil 09850-901;Instituto Tecnológico de Aeronáutica, São José dos Campos, Brazil 12228-900;Escola Politécnica da Universidade de São Paulo, São Paulo, Brazil 05508-900
Venue:
Journal of Heuristics
Year:
2008

Citing 15
Cited 7

Dynamic programming: deterministic and stochastic models

Dynamic programming: deterministic and stochastic models
Using Occupancy Grids for Mobile Robot Perception and Navigation

Computer
Integrated architecture for learning, planning, and reacting based on approximating dynamic programming

Proceedings of the seventh international conference (1990) on Machine learning
Efficient learning and planning within the Dyna framework

Adaptive Behavior
Prioritized Sweeping: Reinforcement Learning with Less Data and Less Time

Machine Learning
Artificial intelligence: a modern approach

Artificial intelligence: a modern approach
The saphira architecture for autonomous mobile robots

Artificial intelligence and mobile robots
Robust Monte Carlo localization for mobile robots

Artificial Intelligence
Machine Learning

Machine Learning
Continuous-Action Q-Learning

Machine Learning
Structure in the Space of Value Functions

Machine Learning
Variable Resolution Discretization in Optimal Control

Machine Learning
Learning to Predict by the Methods of Temporal Differences

Machine Learning
Accelerating reinforcement learning by composing solutions of automatically identified subtasks

Journal of Artificial Intelligence Research
Reinforcement learning: a survey

Journal of Artificial Intelligence Research

Improving Reinforcement Learning by Using Case Based Heuristics

ICCBR '09 Proceedings of the 8th International Conference on Case-Based Reasoning: Case-Based Reasoning Research and Development
Case-Based Multiagent Reinforcement Learning: Cases as Heuristics for Selection of Actions

Proceedings of the 2010 conference on ECAI 2010: 19th European Conference on Artificial Intelligence
Market-based dynamic task allocation using heuristically accelerated reinforcement learning

EPIA'11 Proceedings of the 15th Portugese conference on Progress in artificial intelligence
Including cognitive biases and distance-based rewards in a connectionist model of complex problem solving

Neural Networks
Stochastic abstract policies for knowledge transfer in robotic navigation tasks

MICAI'11 Proceedings of the 10th Mexican international conference on Advances in Artificial Intelligence - Volume Part I
Using cases as heuristics in reinforcement learning: a transfer learning application

IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Two
Backward Q-learning: The combination of Sarsa algorithm and Q-learning

Engineering Applications of Artificial Intelligence

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper investigates how to make improved action selection for online policy learning in robotic scenarios using reinforcement learning (RL) algorithms. Since finding control policies using any RL algorithm can be very time consuming, we propose to combine RL algorithms with heuristic functions for selecting promising actions during the learning process. With this aim, we investigate the use of heuristics for increasing the rate of convergence of RL algorithms and contribute with a new learning algorithm, Heuristically Accelerated Q-learning (HAQL), which incorporates heuristics for action selection to the Q-Learning algorithm. Experimental results on robot navigation show that the use of even very simple heuristic functions results in significant performance enhancement of the learning rate.