Artificial Intelligence
A model for reasoning about persistence and causation
Computational Intelligence
Proceedings of the first international conference on Principles of knowledge representation and reasoning
Handbook of theoretical computer science (vol. B)
Learning to act using real-time dynamic programming
Artificial Intelligence - Special volume on computational research on interaction and agency, part 1
Efficient checking of temporal integrity constraints using bounded history encoding
ACM Transactions on Database Systems (TODS)
Planning under time constraints in stochastic domains
Artificial Intelligence - Special volume on planning and scheduling
An algorithm for probabilistic planning
Artificial Intelligence - Special volume on planning and scheduling
Using temporal logics to express search control knowledge for planning
Artificial Intelligence
Stochastic dynamic programming with factored representations
Artificial Intelligence
Artificial Intelligence
LAO: a heuristic search algorithm that finds solutions with loops
Artificial Intelligence - Special issue on heuristic search in artificial intelligence
Planning for temporally extended goals
Annals of Mathematics and Artificial Intelligence
On the Relation of Programs and Computations to Models of Temporal Logic
Temporal Logic in Specification
Proceedings of the Conference on Logic of Programs
Planning with a language for extended goals
Eighteenth national conference on Artificial intelligence
Symbolic heuristic search for factored Markov decision processes
Eighteenth national conference on Artificial intelligence
The FF planning system: fast plan generation through heuristic search
Journal of Artificial Intelligence Research
The first probabilistic track of the international planning competition
Journal of Artificial Intelligence Research
mGPT: a probabilistic planner based on heuristic search
Journal of Artificial Intelligence Research
Engineering a conformant probabilistic planner
Journal of Artificial Intelligence Research
Automated verification: graphs, logic, and automata
IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Planning as model checking for extended goals in non-deterministic domains
IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 1
AAAI'96 Proceedings of the thirteenth national conference on Artificial intelligence - Volume 2
Structured solution methods for non-Markovian decision processes
AAAI'97/IAAI'97 Proceedings of the fourteenth national conference on artificial intelligence and ninth conference on Innovative applications of artificial intelligence
SPUDD: stochastic planning using decision diagrams
UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence
Anytime state-based solution methods for decision processes with non-Markovian rewards
UAI'02 Proceedings of the Eighteenth conference on Uncertainty in artificial intelligence
Symbolic generalization for on-line planning
UAI'03 Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence
Implementation and comparison of solution methods for decision processes with non-markovian rewards
UAI'03 Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence
Using α-ctl to Specify Complex Planning Goals
WoLLIC '08 Proceedings of the 15th international workshop on Logic, Language, Information and Computation
Practical solution techniques for first-order MDPs
Artificial Intelligence
Non-monotonic temporal logics that facilitate elaboration tolerant revision of goals
AAAI'08 Proceedings of the 23rd national conference on Artificial intelligence - Volume 1
Engineering a conformant probabilistic planner
Journal of Artificial Intelligence Research
A heuristic search approach to planning with continuous resources in stochastic domains
Journal of Artificial Intelligence Research
Electronic Notes in Theoretical Computer Science (ENTCS)
LTL Goal Specifications Revisited
Proceedings of the 2010 conference on ECAI 2010: 19th European Conference on Artificial Intelligence
Shortest stochastic path with risk sensitive evaluation
MICAI'12 Proceedings of the 11th Mexican international conference on Advances in Artificial Intelligence - Volume Part I
Hi-index | 0.05 |
A decision process in which rewards depend on history rather than merely on the current state is called a decision process with non-Markovian rewards (NMRDP). In decision-theoretic planning, where many desirable behaviours are more naturally expressed as properties of execution sequences rather than as properties of states, NMRDPs form a more natural model than the commonly adopted fully Markovian decision process (MDP) model. While the more tractable solution methods developed for MDPs do not directly apply in the presence of non-Markovian rewards, a number of solution methods for NMRDPs have been proposed in the literature. These all exploit a compact specification of the non-Markovian reward function in temporal logic, to automatically translate the NMRDP into an equivalent MDP which is solved using efficient MDP solution methods. This paper presents NMRDPP(Non-Markovian Reward Decision Process Planner), a software platform for the development and experimentation of methods for decision-theoretic planning with non-Markovian rewards. The current version of NMRDPP implements, under a single interface, a family of methods based on existing as well as new approaches which we describe in detail. These include dynamic programming, heuristic search, and structured methods. Using NMRDPP, we compare the methods and identify certain problem features that affect their performance. NMRDPP's treatment of non-Markovian rewards is inspired by the treatment of domain-specific search control knowledge in the TLPlan planner, which it incorporates as a special case. In the First International Probabilistic Planning Competition, NMRDPP was able to compete and perform well in both the domain-independent and hand-coded tracks, using search control knowledge in the latter.