Learning to solve problems by searching for macro-operators
Learning to solve problems by searching for macro-operators
Dynamic programming: deterministic and stochastic models
Dynamic programming: deterministic and stochastic models
IEEE Transactions on Systems, Man and Cybernetics
Statecharts: A visual formalism for complex systems
Science of Computer Programming
A model for reasoning about persistence and causation
Computational Intelligence
Practical Issues in Temporal Difference Learning
Machine Learning
Technical Note: \cal Q-Learning
Machine Learning
TD-Gammon, a self-teaching backgammon program, achieves master-level play
Neural Computation
Learning to act using real-time dynamic programming
Artificial Intelligence - Special volume on computational research on interaction and agency, part 1
Some studies in machine learning using the game of checkers
Computers & thought
Average reward reinforcement learning: foundations, algorithms, and empirical results
Machine Learning - Special issue on reinforcement learning
Planning and acting in partially observable stochastic domains
Artificial Intelligence
Xavier: a robot navigation architecture based on partially observable Markov decision process models
Artificial intelligence and mobile robots
Learning hierarchical control structures for multiple tasks and changing environments
Proceedings of the fifth international conference on simulation of adaptive behavior on From animals to animats 5
Reinforcement learning with hierarchies of machines
NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
Multi-time models for temporally abstract planning
NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
Elevator Group Control Using Multiple Reinforcement Learning Agents
Machine Learning
Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning
Artificial Intelligence
The Hierarchical Hidden Markov Model: Analysis and Applications
Machine Learning
Transition network grammars for natural language analysis
Communications of the ACM
Hierarchical multi-agent reinforcement learning
Proceedings of the fifth international conference on Autonomous agents
Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence
Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence
Introduction to Stochastic Dynamic Programming: Probability and Mathematical
Introduction to Stochastic Dynamic Programming: Probability and Mathematical
Introduction to Reinforcement Learning
Introduction to Reinforcement Learning
Neuro-Dynamic Programming
Singular Perturbation Methods in Control: Analysis and Design
Singular Perturbation Methods in Control: Analysis and Design
A Heuristic Approach to the Discovery of Macro-Operators
Machine Learning
Theoretical Results on Reinforcement Learning with Temporally Abstract Options
ECML '98 Proceedings of the 10th European Conference on Machine Learning
Scaling Reinforcement Learning toward RoboCup Soccer
ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Automatic Discovery of Subgoals in Reinforcement Learning using Diverse Density
ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Discovering Hierarchy in Reinforcement Learning with HEXQ
ICML '02 Proceedings of the Nineteenth International Conference on Machine Learning
Integrating Experimentation and Guidance in Relational Reinforcement Learning
ICML '02 Proceedings of the Nineteenth International Conference on Machine Learning
Scaling Reinforcement Learning Algorithms by Learning Variable Temporal Resolution Models
ML '92 Proceedings of the Ninth International Workshop on Machine Learning
Continuous-Time Hierarchical Reinforcement Learning
ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Lyapunov-Constrained Action Sets for Reinforcement Learning
ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
The Complexity of Decentralized Control of Markov Decision Processes
UAI '00 Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence
Decision-Theoretic Planning with Concurrent Temporally Extended Actions
UAI '01 Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence
Localizing Search in Reinforcement Learning
Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence
Achieving Artificial Intelligence through Building Robots
Achieving Artificial Intelligence through Building Robots
Reinforcement learning with selective perception and hidden state
Reinforcement learning with selective perception and hidden state
Large-scale dynamic optimization using teams of reinforcement learning agents
Large-scale dynamic optimization using teams of reinforcement learning agents
Hierarchical control and learning for markov decision processes
Hierarchical control and learning for markov decision processes
Temporal abstraction in reinforcement learning
Temporal abstraction in reinforcement learning
Autonomous discovery of temporal abstractions from interaction with an environment
Autonomous discovery of temporal abstractions from interaction with an environment
Hierarchical learning and planning in partially observable markov decision processes
Hierarchical learning and planning in partially observable markov decision processes
Lyapunov design for safe reinforcement learning
The Journal of Machine Learning Research
Hierarchical reinforcement learning with the MAXQ value function decomposition
Journal of Artificial Intelligence Research
Learning topological maps with weak local odometric information
IJCAI'97 Proceedings of the Fifteenth international joint conference on Artifical intelligence - Volume 2
Reinforcement learning: a survey
Journal of Artificial Intelligence Research
Tractable inference for complex stochastic processes
UAI'98 Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence
Dynamic abstraction in reinforcement learning via clustering
ICML '04 Proceedings of the twenty-first international conference on Machine learning
Hierarchical Reinforcement Learning in Communication-Mediated Multiagent Coordination
AAMAS '04 Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems - Volume 3
Basic Ideas for Event-Based Optimization of Markov Systems
Discrete Event Dynamic Systems
An Evolutionary Dynamical Analysis of Multi-Agent Learning in Iterated Games
Autonomous Agents and Multi-Agent Systems
Fast direct policy evaluation using multiscale analysis of Markov diffusion processes
ICML '06 Proceedings of the 23rd international conference on Machine learning
Kernel rewards regression: an information efficient batch policy iteration approach
AIA'06 Proceedings of the 24th IASTED international conference on Artificial intelligence and applications
Application of stochastic learning automata to intelligent vehicle control
ICCOMP'07 Proceedings of the 11th WSEAS International Conference on Computers
An ASML model for an intelligent vehicle control system
ICCOMP'07 Proceedings of the 11th WSEAS International Conference on Computers
Scaling ant colony optimization with hierarchical reinforcement learning partitioning
Proceedings of the 10th annual conference on Genetic and evolutionary computation
Hierarchical model-based reinforcement learning: R-max + MAXQ
Proceedings of the 25th international conference on Machine learning
Hierarchical Average Reward Reinforcement Learning
The Journal of Machine Learning Research
Hierarchical Co-evolution of Cooperating Agents Acting in the Brain-Arena
Adaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems
The utility of temporal abstraction in reinforcement learning
Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 1
Reinforcement learning for problems with symmetrical restricted states
Robotics and Autonomous Systems
Adaptiveness in Agent Communication: Application and Adaptation of Conversation Patterns
Agent Communication II
Multi-robot Cooperation Based on Hierarchical Reinforcement Learning
ICCS '07 Proceedings of the 7th international conference on Computational Science, Part III: ICCS 2007
Robotics and Autonomous Systems
Partial Order Hierarchical Reinforcement Learning
AI '08 Proceedings of the 21st Australasian Joint Conference on Artificial Intelligence: Advances in Artificial Intelligence
A new evolutionary reinforcement scheme for stochastic learning automata
ICCOMP'08 Proceedings of the 12th WSEAS international conference on Computers
Reinforcement Learning: A Tutorial Survey and Recent Advances
INFORMS Journal on Computing
Automatic control based on wasp behavioral model and stochastic learning automata
MAMECTIS'08 Proceedings of the 10th WSEAS international conference on Mathematical methods, computational techniques and intelligent systems
Junction Tree Factored Particle Inference Algorithm for Multi-Agent Dynamic Influence Diagrams
FAW '09 Proceedings of the 3d International Workshop on Frontiers in Algorithmics
Learning Representation and Control in Markov Decision Processes: New Frontiers
Foundations and Trends® in Machine Learning
Advice generation from observed execution: abstract Markov decision process learning
AAAI'04 Proceedings of the 19th national conference on Artifical intelligence
Compositional Models for Reinforcement Learning
ECML PKDD '09 Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases: Part I
A computational model of the cerebral cortex
AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 2
Evaluation of a hierarchical reinforcement learning spoken dialogue system
Computer Speech and Language
Refining the execution of abstract actions with learned action models
Journal of Artificial Intelligence Research
Building portable options: skill transfer in reinforcement learning
IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Preference-Aware Web Service Composition Using Hierarchical Reinforcement Learning
WI-IAT '09 Proceedings of the 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology - Volume 03
Automatic abstraction in reinforcement learning using data mining techniques
Robotics and Autonomous Systems
Optimized execution of action chains using learned performance models of abstract actions
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Efficient skill learning using abstraction selection
IJCAI'09 Proceedings of the 21st international jont conference on Artifical intelligence
Learning relational options for inductive transfer in relational reinforcement learning
ILP'07 Proceedings of the 17th international conference on Inductive logic programming
IEEE Transactions on Evolutionary Computation
Basis function construction for hierarchical reinforcement learning
Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1
Optimizing a new nonlinear reinforcement scheme with Breeder genetic algorithm
NN'10/EC'10/FS'10 Proceedings of the 11th WSEAS international conference on nural networks and 11th WSEAS international conference on evolutionary computing and 11th WSEAS international conference on Fuzzy systems
Hierarchical reinforcement learning for adaptive text generation
INLG '10 Proceedings of the 6th International Natural Language Generation Conference
TeXDYNA: hierarchical reinforcement learning in factored MDPs
SAB'10 Proceedings of the 11th international conference on Simulation of adaptive behavior: from animals to animats
Distributed Reinforcement Learning for Coordinate Multi-Robot Foraging
Journal of Intelligent and Robotic Systems
Autonomous discovery of subgoals using acyclic state trajectories
ICICA'10 Proceedings of the First international conference on Information computing and applications
A nonlinear reinforcement scheme for stochastic learning automata
MMACTEE'06 Proceedings of the 8th WSEAS international conference on Mathematical methods and computational techniques in electrical engineering
Heliza: talking dirty to the attackers
Journal in Computer Virology
Probabilistic generalization of simple grammars and its application to reinforcement learning
ALT'06 Proceedings of the 17th international conference on Algorithmic Learning Theory
Reinforcement learning in mirrorbot
ICANN'05 Proceedings of the 15th international conference on Artificial Neural Networks: biological Inspirations - Volume Part I
Grey reinforcement learning for incomplete information processing
TAMC'06 Proceedings of the Third international conference on Theory and Applications of Models of Computation
Generating admissible heuristics by abstraction for search in stochastic domains
SARA'05 Proceedings of the 6th international conference on Abstraction, Reformulation and Approximation
An overview of cooperative and competitive multiagent learning
LAMAS'05 Proceedings of the First international conference on Learning and Adaption in Multi-Agent Systems
EWRL'11 Proceedings of the 9th European conference on Recent Advances in Reinforcement Learning
Transfer in reinforcement learning via shared features
The Journal of Machine Learning Research
Induction and learning of finite-state controllers from simulation
Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 3
A modular hierarchical reinforcement learning algorithm
ICIC'12 Proceedings of the 8th international conference on Intelligent Computing Theories and Applications
Automatic generation and learning of finite-state controllers
AIMSA'12 Proceedings of the 15th international conference on Artificial Intelligence: methodology, systems, and applications
A hierarchical representation policy iteration algorithm for reinforcement learning
IScIDE'12 Proceedings of the third Sino-foreign-interchange conference on Intelligent Science and Intelligent Data Engineering
Neuroevolution results in emergence of short-term memory in multi-goal environment
Proceedings of the 15th annual conference on Genetic and evolutionary computation
Proceedings of the 2nd Workshop on Machine Learning for Interactive Systems: Bridging the Gap Between Perception, Action and Communication
Machine learning for interactive systems and robots: a brief introduction
Proceedings of the 2nd Workshop on Machine Learning for Interactive Systems: Bridging the Gap Between Perception, Action and Communication
A novel reinforcement learning architecture for continuous state and action spaces
Advances in Artificial Intelligence
Integrated task and motion planning in belief space
International Journal of Robotics Research
Soft robotics: the next generation of intelligent machines
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Reinforcement learning algorithms with function approximation: Recent advances and applications
Information Sciences: an International Journal
Hi-index | 0.00 |
Reinforcement learning is bedeviled by the curse of dimensionality: the number of parameters to be learned grows exponentially with the size of any compact encoding of a state. Recent attempts to combat the curse of dimensionality have turned to principled ways of exploiting temporal abstraction, where decisions are not required at each step, but rather invoke the execution of temporally-extended activities which follow their own policies until termination. This leads naturally to hierarchical control architectures and associated learning algorithms. We review several approaches to temporal abstraction and hierarchical organization that machine learning researchers have recently developed. Common to these approaches is a reliance on the theory of semi-Markov decision processes, which we emphasize in our review. We then discuss extensions of these ideas to concurrent activities, multiagent coordination, and hierarchical memory for addressing partial observability. Concluding remarks address open challenges facing the further development of reinforcement learning in a hierarchical setting.