A model for reasoning about persistence and causation
Computational Intelligence
C4.5: programs for machine learning
C4.5: programs for machine learning
Abstraction and approximate decision-theoretic planning
Artificial Intelligence
Qualitative reverse engineering
ICML '02 Proceedings of the Nineteenth International Conference on Machine Learning
Reinforcement Learning with Factored States and Actions
The Journal of Machine Learning Research
Dynamic programming for structured continuous Markov decision problems
UAI '04 Proceedings of the 20th conference on Uncertainty in artificial intelligence
Lazy approximation for solving continuous finite-horizon MDPs
AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 3
Efficient reinforcement learning in factored MDPs
IJCAI'99 Proceedings of the 16th international joint conference on Artificial intelligence - Volume 2
Variable resolution discretization for high-accuracy solutions of optimal control problems
IJCAI'99 Proceedings of the 16th international joint conference on Artificial intelligence - Volume 2
Model minimization in Markov decision processes
AAAI'97/IAAI'97 Proceedings of the fourteenth national conference on artificial intelligence and ninth conference on Innovative applications of artificial intelligence
Continuous value function approximation for sequential bidding policies
UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence
Policy-contingent abstraction for robust robot control
UAI'03 Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence
| Hi-index | 0.00 |
Markov decision processes (MDPs) have developed as a standard for representing uncertainty in decision-theoretic planning. However, MDPs require an explicit representation of the state space and the probabilistic transition model which, in continuous or hybrid continuous-discrete domains, are not always easy to define. Even when this representation is available, the size of the state space and the number of state variables to consider in the transition function may be such that the resulting MDP cannot be solved using traditional techniques. In this paper a reward-based abstraction for solving hybrid MDPs is presented. In the proposed method, we gather information about the rewards and the dynamics of the system by exploring the environment. This information is used to build a decision tree (C4.5) representing a small set of abstract states with equivalent rewards, and then is used to learn a probabilistic transition function using a Bayesian networks learning algorithm (K2). The system output is a problem specification ready for its solution with traditional dynamic programming algorithms. We have tested our abstract MDP model approximation in real-world problem domains. We present the results in terms of the models learned and their solutions for different configurations showing that our approach produces fast solutions with satisfying policies.