The implementation of Q-learning for problems in continuous state and action space using SOM-based fuzzy systems

Authors:
Mu-Chun Su;Lu-Yu Chen;De-Yuan Huang
Affiliations:
Department of Computer Science & Information Engineering, National Central University, Taiwan, R.O.C.;Department of Computer Science & Information Engineering, National Central University, Taiwan, R.O.C.;Department of Computer Science & Information Engineering, National Central University, Taiwan, R.O.C.
Venue:
ICCOMP'06 Proceedings of the 10th WSEAS international conference on Computers
Year:
2006

Citing 15
Cited 0

Induction: processes of inference, learning, and discovery

Induction: processes of inference, learning, and discovery
Technical Note: \cal Q-Learning

Machine Learning
Transfer of Learning by Composing Solutions of Elemental Sequential Tasks

Machine Learning
Reinforcement learning with hidden states

Proceedings of the second international conference on From animals to animats 2 : simulation of adaptive behavior: simulation of adaptive behavior
Experiments with reinforcement learning in problems with continuous state and action spaces

Adaptive Behavior
Genetic Algorithms in Search, Optimization and Machine Learning

Genetic Algorithms in Search, Optimization and Machine Learning
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Reinforcement Learning Agents

Artificial Intelligence Review
Learning to Predict by the Methods of Temporal Differences

Machine Learning
Q-Learning in Continuous State and Action Spaces

AI '99 Proceedings of the 12th Australian Joint Conference on Artificial Intelligence: Advanced Topics in Artificial Intelligence
Reinforcement learning: a survey

Journal of Artificial Intelligence Research
A new approach to fuzzy classifier systems and its application in self-generating neuro-fuzzy systems

Neurocomputing
Expertness based cooperative Q-learning

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Online learning control by association and reinforcement

IEEE Transactions on Neural Networks
Learning to trade via direct reinforcement

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

In reinforcement learning, there is no supervisor to critically judge the chosen action at each step. The learning is through a trial-and-error procedure interacting with a dynamic environment. Q-learning is one popular approach to reinforcement learning. It is widely applied to problems with discrete states and actions and usually implemented by a look-up table where each item corresponds to a combination of a state and an action. However, the look-up table implementation of Q-learning fails in problems with continuous state and action space because an exhaustive enumeration of all state-action pairs is impossible. In this paper, an implementation of Q-learning for solving problems with continuous state and action space using SOM-based fuzzy systems is proposed. Simulations of training a robot to complete two different tasks are used to demonstrate the effectiveness of the proposed approach. Reinforcement learning usually is a slow process. In order to accelerate the learning procedure, a hybrid approach which integrates the advantages of the ideas of hierarchical learning and the progressive learning to decompose a complex task into simple elementary tasks is proposed.