Adaptive planning for markov decision processes with uncertain transition models via incremental feature dependency discovery

Authors:
N. Kemal Ure;Alborz Geramifard;Girish Chowdhary;Jonathan P. How
Affiliations:
Laboratory for Information and Decision Systems, (MIT) Massachusetts Institute of Technology, Cambridge, MA;Laboratory for Information and Decision Systems, (MIT) Massachusetts Institute of Technology, Cambridge, MA;Laboratory for Information and Decision Systems, (MIT) Massachusetts Institute of Technology, Cambridge, MA;Laboratory for Information and Decision Systems, (MIT) Massachusetts Institute of Technology, Cambridge, MA
Venue:
ECML PKDD'12 Proceedings of the 2012 European conference on Machine Learning and Knowledge Discovery in Databases - Volume Part II
Year:
2012

Citing 18
Cited 1

Prioritized Sweeping: Reinforcement Learning with Less Data and Less Time

Machine Learning
Convergence analysis of gradient descent stochastic algorithms

Journal of Optimization Theory and Applications
Markov Decision Processes: Discrete Stochastic Dynamic Programming

Markov Decision Processes: Discrete Stochastic Dynamic Programming
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Artificial Intelligence: A Modern Approach

Artificial Intelligence: A Modern Approach
R-max - a general polynomial time algorithm for near-optimal reinforcement learning

The Journal of Machine Learning Research
Robust Dynamic Programming

Mathematics of Operations Research
Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning)

Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning)
Robust Control of Markov Decision Processes with Uncertain Transition Matrices

Operations Research
Dynamic Programming and Optimal Control, Vol. II

Dynamic Programming and Optimal Control, Vol. II
Learning to act using real-time dynamic programming

Artificial Intelligence
Incremental learning of statistical motion patterns with growing hidden Markov models

IEEE Transactions on Intelligent Transportation Systems
Percentile Optimization for Markov Decision Processes with Parameter Uncertainty

Operations Research
A Bayesian sampling approach to exploration in reinforcement learning

UAI '09 Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence
Reinforcement Learning and Dynamic Programming Using Function Approximators

Reinforcement Learning and Dynamic Programming Using Function Approximators
A Bayesian nonparametric approach to modeling motion patterns

Autonomous Robots
Online Bayesian estimation of transition probabilities for Markovian jump systems

IEEE Transactions on Signal Processing
The Kernel Least-Mean-Square Algorithm

IEEE Transactions on Signal Processing

Distributed Learning for Planning Under Uncertainty Problems with Heterogeneous Teams

Journal of Intelligent and Robotic Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

Solving large scale sequential decision making problems without prior knowledge of the state transition model is a key problem in the planning literature. One approach to tackle this problem is to learn the state transition model online using limited observed measurements. We present an adaptive function approximator (incremental Feature Dependency Discovery (iFDD)) that grows the set of features online to approximately represent the transition model. The approach leverages existing feature-dependencies to build a sparse representation of the state transition model. Theoretical analysis and numerical simulations in domains with state space sizes varying from thousands to millions are used to illustrate the benefit of using iFDD for incrementally building transition models in a planning framework.