Technical Note: \cal Q-Learning
Machine Learning
RoboCup: The Robot World Cup Initiative
AGENTS '97 Proceedings of the first international conference on Autonomous agents
Dynamic Programming and Optimal Control
Dynamic Programming and Optimal Control
Programmable self-assembly using biologically-inspired multiagent control
Proceedings of the first international joint conference on Autonomous agents and multiagent systems: part 1
Introduction to Reinforcement Learning
Introduction to Reinforcement Learning
Reinforcement Learning in the Multi-Robot Domain
Autonomous Robots
Multiagent Systems: A Survey from a Machine Learning Perspective
Autonomous Robots
ICML '99 Proceedings of the Sixteenth International Conference on Machine Learning
Collaborating With A Genetic Programming System To Generate Modular Robotic Code
GECCO '02 Proceedings of the Genetic and Evolutionary Computation Conference
PEGASUS: A policy search method for large MDPs and POMDPs
UAI '00 Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence
Reinforcement learning by policy search
Reinforcement learning by policy search
Least-squares policy iteration
The Journal of Machine Learning Research
Collaborative Multiagent Reinforcement Learning by Payoff Propagation
The Journal of Machine Learning Research
Infinite-horizon policy-gradient estimation
Journal of Artificial Intelligence Research
Dimensionality effects on the Markov property in shape memory alloy hysteretic environment
SMC'09 Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics
Roombots: reconfigurable robots for adaptive furniture
IEEE Computational Intelligence Magazine
A compositional framework for programming stochastically interacting robots
International Journal of Robotics Research
On-line assembly planning for stochastically reconfigurable systems
International Journal of Robotics Research
A combined reactive and reinforcement learning controller for an autonomous tracked vehicle
Robotics and Autonomous Systems
Robotics and Autonomous Systems
| Hi-index | 0.00 |
Designing distributed controllers for self-reconfiguring modular robots has been consistently challenging. We have developed a reinforcement learning approach which can be used both to automate controller design and to adapt robot behavior on-line. In this paper, we report on our study of reinforcement learning in the domain of self-reconfigurable modular robots: the underlying assumptions, the applicable algorithms and the issues of partial observability, large search spaces and local optima. We propose and validate experimentally in simulation a number of techniques designed to address these and other scalability issues that arise in applying machine learning to distributed systems such as modular robots. We discuss ways to make learning faster, more robust and amenable to on-line application by giving scaffolding to the learning agents in the form of policy representation, structured experience and additional information. With enough structure modular robots can run learning algorithms to both automate the generation of distributed controllers, and adapt to the changing environment and deliver on the self-organization promise with less interference from human designers, programmers and operators.