Using control theory for analysis of reinforcement learning and optimal policy properties in grid-world problems

Authors:
S. Mostapha Kalami Heris;Mohammad-Bagher Naghibi Sistani;Naser Pariz
Affiliations:
Control Engineering Department, Faculty of Electrical Engineering, K. N. Toosi University of Technology, Tehran, Iran;Electrical Engineering Department, Faculty of Engineering, Ferdowsi University of Mashhad, Mashhad, Iran;Electrical Engineering Department, Faculty of Engineering, Ferdowsi University of Mashhad, Mashhad, Iran
Venue:
ICIC'09 Proceedings of the Intelligent computing 5th international conference on Emerging intelligent computing technology and applications
Year:
2009

Citing 19
Cited 0

Discrete-time control systems (2nd ed.)

Discrete-time control systems (2nd ed.)
An improved policy iteration algorithm for partially observable MDPs

NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
Bounded-parameter Markov decision process

Artificial Intelligence
On the existence of fixed points for approximate value iteration and temporal-difference learning

Journal of Optimization Theory and Applications
Markov Decision Processes: Discrete Stochastic Dynamic Programming

Markov Decision Processes: Discrete Stochastic Dynamic Programming
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Neuro-Dynamic Programming

Neuro-Dynamic Programming
Monotone Optimal Policies for a Transient Queueing Staffing Problem

Operations Research
Structural Properties of Stochastic Dynamic Programs

Operations Research
Reusing Old Policies to Accelerate Learning on New MDPs TITLE2:

Reusing Old Policies to Accelerate Learning on New MDPs TITLE2:
Exact and approximate algorithms for partially observable markov decision processes

Exact and approximate algorithms for partially observable markov decision processes
Integration of partially observable markov decision processes and reinforcement learning for simulated robot navigation

Integration of partially observable markov decision processes and reinforcement learning for simulated robot navigation
Approximate solutions to markov decision processes

Approximate solutions to markov decision processes
Reinforcement learning for factored markov decision processes

Reinforcement learning for factored markov decision processes
Handbook of Learning and Approximate Dynamic Programming (IEEE Press Series on Computational Intelligence)

Handbook of Learning and Approximate Dynamic Programming (IEEE Press Series on Computational Intelligence)
Simulation-based Algorithms for Markov Decision Processes (Communications and Control Engineering)

Simulation-based Algorithms for Markov Decision Processes (Communications and Control Engineering)
Dynamic Programming: A Computational Tool (Studies in Computational Intelligence)

Dynamic Programming: A Computational Tool (Studies in Computational Intelligence)
Speeding up the convergence of value iteration in partially observable Markov decision processes

Journal of Artificial Intelligence Research
Reinforcement learning: a survey

Journal of Artificial Intelligence Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

Markov Decision Process (MDP) has enormous applications in science, engineering, economics and management. Most of decision processes have Markov property and can be modeled as MDP. Reinforcement Learning (RL) is an approach to deal with MDPs. RL methods are based on Dynamic Programming (DP) algorithms, such as Policy Evaluation, Policy Iteration and Value Iteration. In this paper, policy evaluation algorithm is represented in the form of a discrete-time dynamical system. Hence, using Discrete-Time Control methods, behavior of agent and properties of various policies, can be analyzed. The general case of grid-world problems is addressed, and some important results are obtained for this type of problems as a theorem. For example, equivalent system of an optimal policy for a grid-world problem is dead-beat.