Efficient non-linear control through neuroevolution

Authors:
Faustino Gomez;Jürgen Schmidhuber;Risto Miikkulainen
Affiliations:
Dalle Molle Institute for Artificial Intelligence (IDSIA), Lugano, CH;Dalle Molle Institute for Artificial Intelligence (IDSIA), Lugano, CH;Department of Computer Sciences, University of Texas, Austin, TX
Venue:
ECML'06 Proceedings of the 17th European conference on Machine Learning
Year:
2006

Citing 10
Cited 38

Technical Note: \cal Q-Learning

Machine Learning
Experiments with reinforcement learning in problems with continuous state and action spaces

Adaptive Behavior
Gradient descent for general reinforcement learning

Proceedings of the 1998 conference on Advances in neural information processing systems II
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Evolving Neural Control Systems

IEEE Expert: Intelligent Systems and Their Applications
Evolving neural networks through augmenting topologies

Evolutionary Computation
Symbiotic Evolution of Neural Networks in Sequential Decision Tasks

Symbiotic Evolution of Neural Networks in Sequential Decision Tasks
Robust non-linear control through neuroevolution

Robust non-linear control through neuroevolution
Completely Derandomized Self-Adaptation in Evolution Strategies

Evolutionary Computation
Learning finite-state controllers for partially observable environments

UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence

Evolving neural networks

Proceedings of the 9th annual conference companion on Genetic and evolutionary computation
A common genetic encoding for both direct and indirect encodings of networks

Proceedings of the 9th annual conference on Genetic and evolutionary computation
Restricted gradient-descent algorithm for value-function approximation in reinforcement learning

Artificial Intelligence
Modular neuroevolution for multilegged locomotion

Proceedings of the 10th annual conference on Genetic and evolutionary computation
Accelerating neuroevolutionary methods using a Kalman filter

Proceedings of the 10th annual conference on Genetic and evolutionary computation
Evolving neural networks for fractured domains

Proceedings of the 10th annual conference on Genetic and evolutionary computation
Theoretical Advantages of Lenient Learners: An Evolutionary Game Theoretic Perspective

The Journal of Machine Learning Research
Analysis of an evolutionary reinforcement learning method in a multiagent domain

Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 1
The 2007 IEEE CEC simulated car racing competition

Genetic Programming and Evolvable Machines
Evolving Efficient Connection for the Design of Artificial Neural Networks

ICANN '08 Proceedings of the 18th international conference on Artificial Neural Networks, Part II
Fitness Expectation Maximization

Proceedings of the 10th international conference on Parallel Problem Solving from Nature: PPSN X
Evolution Strategies for Direct Policy Search

Proceedings of the 10th international conference on Parallel Problem Solving from Nature: PPSN X
Evolving Neural Networks for Online Reinforcement Learning

Proceedings of the 10th international conference on Parallel Problem Solving from Nature: PPSN X
EANT+KALMAN: An Efficient Reinforcement Learning Method for Continuous State Partially Observable Domains

KI '08 Proceedings of the 31st annual German conference on Advances in Artificial Intelligence
Variable Metric Reinforcement Learning Methods Applied to the Noisy Mountain Car Problem

Recent Advances in Reinforcement Learning
Efficient covariance matrix update for variable metric evolution strategies

Machine Learning
Using promoters and functional introns in genetic algorithms for neuroevolutionary learning in non-stationary problems

Neurocomputing
2009 Special Issue: Evolving neural networks for strategic decision-making problems

Neural Networks
Stochastic search using the natural gradient

ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Neuroevolutionary reinforcement learning for generalized helicopter control

Proceedings of the 11th Annual conference on Genetic and evolutionary computation
NEAT in increasingly non-linear control situations

Proceedings of the 11th Annual Conference Companion on Genetic and Evolutionary Computation Conference: Late Breaking Papers
Neuroevolution strategies for episodic reinforcement learning

Journal of Algorithms
Machine learning for event selection in high energy physics

Engineering Applications of Artificial Intelligence
Knowledge-based recurrent neural networks in Reinforcement Learning

ASC '07 Proceedings of The Eleventh IASTED International Conference on Artificial Intelligence and Soft Computing
Speeding up reinforcement learning using recurrent neural networks in non-Markovian environments

ASC '07 Proceedings of The Eleventh IASTED International Conference on Artificial Intelligence and Soft Computing
Neuro-evolution approaches to collective behavior

CEC'09 Proceedings of the Eleventh conference on Congress on Evolutionary Computation
2006: celebrating 75 years of AI - history and outlook: the next 25 years

50 years of artificial intelligence
Machine learning techniques for selforganizing combustion control

KI'09 Proceedings of the 32nd annual German conference on Advances in artificial intelligence
Critical factors in the empirical performance of temporal difference and evolutionary methods for reinforcement learning

Autonomous Agents and Multi-Agent Systems
Evolving neural networks in compressed weight space

Proceedings of the 12th annual conference on Genetic and evolutionary computation
A NEAT Way for Evolving Echo State Networks

Proceedings of the 2010 conference on ECAI 2010: 19th European Conference on Artificial Intelligence
Theoretical convergence guarantees for cooperative coevolutionary algorithms

Evolutionary Computation
A novel NeuroEvolutionary algorithm: Cartesian genetic programming evolved artificial neural network (CGPANN)

Proceedings of the 8th International Conference on Frontiers of Information Technology
On the deleterious effects of a priori objectives on evolution and representation

Proceedings of the 13th annual conference on Genetic and evolutionary computation
Sequential constant size compressors for reinforcement learning

AGI'11 Proceedings of the 4th international conference on Artificial general intelligence
Adaptive reservoir computing through evolution and learning

Neurocomputing
Learning parameters of linear models in compressed parameter space

ICANN'12 Proceedings of the 22nd international conference on Artificial Neural Networks and Machine Learning - Volume Part II
Fast learning neural networks using Cartesian genetic programming

Neurocomputing

Quantified Score

Hi-index	0.01

Visualization

Abstract

Many complex control problems are not amenable to traditional controller design. Not only is it difficult to model real systems, but often it is unclear what kind of behavior is required. Reinforcement learning (RL) has made progress through direct interaction with the task environment, but it has been difficult to scale it up to large and partially observable state spaces. In recent years, neuroevolution, the artificial evolution of neural networks, has shown promise in tasks with these two properties. This paper introduces a novel neuroevolution method called CoSyNE that evolves networks at the level of weights. In the most extensive comparison of RL methods to date, it was tested in difficult versions of the pole-balancing problem that involve large state spaces and hidden state. CoSyNE was found to be significantly more efficient and powerful than the other methods on these tasks, forming a promising foundation for solving challenging real-world control tasks.