Sequential constant size compressors for reinforcement learning

Authors:
Linus Gisslén;Matt Luciw;Vincent Graziano;Jürgen Schmidhuber
Affiliations:
IDSIA, University of Lugano, Manno-Lugano, Switzerland;IDSIA, University of Lugano, Manno-Lugano, Switzerland;IDSIA, University of Lugano, Manno-Lugano, Switzerland;IDSIA, University of Lugano, Manno-Lugano, Switzerland
Venue:
AGI'11 Proceedings of the 4th international conference on Artificial general intelligence
Year:
2011

Citing 15
Cited 3

Recursive distributed representations

Artificial Intelligence - On connectionist symbol processing
Reinforcement learning in Markovian and non-Markovian environments

NIPS-3 Proceedings of the 1990 conference on Advances in neural information processing systems 3
Back propagation is sensitive to initial conditions

NIPS-3 Proceedings of the 1990 conference on Advances in neural information processing systems 3
Efficient reinforcement learning through symbiotic evolution

Machine Learning - Special issue on reinforcement learning
Natural gradient works efficiently in learning

Neural Computation
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Evolving Neural Control Systems

IEEE Expert: Intelligent Systems and Their Applications
Evolving neural networks through augmenting topologies

Evolutionary Computation
Universal Artificial Intelligence: Sequential Decisions Based On Algorithmic Probability

Universal Artificial Intelligence: Sequential Decisions Based On Algorithmic Probability
Completely Derandomized Self-Adaptation in Evolution Strategies

Evolutionary Computation
Long Short-Term Memory

Neural Computation
Policy Gradient Critics

ECML '07 Proceedings of the 18th European conference on Machine Learning
Solving non-Markovian control tasks with neuroevolution

IJCAI'99 Proceedings of the 16th international joint conference on Artificial intelligence - Volume 2
High dimensions and heavy tails for natural evolution strategies

Proceedings of the 13th annual conference on Genetic and evolutionary computation
Efficient non-linear control through neuroevolution

ECML'06 Proceedings of the 17th European conference on Machine Learning

Unsupervised modeling of partially observable environments

ECML PKDD'11 Proceedings of the 2011 European conference on Machine learning and knowledge discovery in databases - Volume Part I
Incremental slow feature analysis: Adaptive low-complexity slow feature updating from high-dimensional input streams

Neural Computation
Modular value iteration through regional decomposition

AGI'12 Proceedings of the 5th international conference on Artificial General Intelligence

Quantified Score

Hi-index	0.00

Visualization

Abstract

Traditional Reinforcement Learning methods are insufficient for AGIs who must be able to learn to deal with Partially Observable Markov Decision Processes. We investigate a novel method for dealing with this problem: standard RL techniques using as input the hidden layer output of a Sequential Constant-Size Compressor (SCSC). The SCSC takes the form of a sequential Recurrent Auto-Associative Memory, trained through standard back-propagation. Results illustrate the feasibility of this approach -- this system learns to deal with highdimensional visual observations (up to 640 pixels) in partially observable environments where there are long time lags (up to 12 steps) between relevant sensory information and necessary action.