Least-cost flaw repair: a plan refinement strategy for partial-order planning
AAAI'94 Proceedings of the twelfth national conference on Artificial intelligence (vol. 2)
Empirical methods for artificial intelligence
Empirical methods for artificial intelligence
Learning search control knowledge to improve plan quality
Learning search control knowledge to improve plan quality
Fast planning through planning graph analysis
Artificial Intelligence
Extending Graphplan to handle uncertainty and sensing actions
AAAI '98/IAAI '98 Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence
Algorithm performance and problem structure for flow-shop scheduling
AAAI '99/IAAI '99 Proceedings of the sixteenth national conference on Artificial intelligence and the eleventh Innovative applications of artificial intelligence conference innovative applications of artificial intelligence
Artificial Intelligence
Test Case Generation as an AI Planning Problem
Automated Software Engineering
Unifying SAT-based and Graph-based Planning
IJCAI '99 Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence
Extending Planning Graphs to an ADL Subset
ECP '97 Proceedings of the 4th European Conference on Planning: Recent Advances in AI Planning
Exploiting Competitive Planner Performance
ECP '99 Proceedings of the 5th European Conference on Planning: Recent Advances in AI Planning
Planning as Heuristic Search: New Results
ECP '99 Proceedings of the 5th European Conference on Planning: Recent Advances in AI Planning
A Computational Model of Skill Acquisition
A Computational Model of Skill Acquisition
Handling of Conditional Effects and Negative Goals in IPP
Handling of Conditional Effects and Negative Goals in IPP
Learning to improve both efficiency and quality of planning
IJCAI'97 Proceedings of the Fifteenth international joint conference on Artifical intelligence - Volume 2
The FF planning system: fast plan generation through heuristic search
Journal of Artificial Intelligence Research
Accelerating partial-order planners: some techniques for effective search control and pruning
Journal of Artificial Intelligence Research
Flaw selection strategies for partial-order planning
Journal of Artificial Intelligence Research
Efficient implementation of the plan graph in STAN
Journal of Artificial Intelligence Research
Local search topology in planning benchmarks: an empirical analysis
IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 1
Comparison of methods for improving search efficiency in a partial-order planner
IJCAI'95 Proceedings of the 14th international joint conference on Artificial intelligence - Volume 2
Applied Artificial Intelligence
Learning from planner performance
Artificial Intelligence
Evaluating AI planning for service composition in smart environments
Proceedings of the 7th International Conference on Mobile and Ubiquitous Multimedia
The 3rd international planning competition: results and analysis
Journal of Artificial Intelligence Research
Engineering benchmarks for planning: the domains used in the deterministic part of IPC-4
Journal of Artificial Intelligence Research
Moving containers in small terminal as STRIPS planning problem
SMO'09 Proceedings of the 9th WSEAS international conference on Simulation, modelling and optimization
Scoring functions of approximation of STRIPS planning by linear programming
SMO'09 Proceedings of the 9th WSEAS international conference on Simulation, modelling and optimization
Scoring functions of approximation of STRIPS planning by linear programming - block world example
WSEAS Transactions on Computers
ICS'10 Proceedings of the 14th WSEAS international conference on Systems: part of the 14th WSEAS CSCC multiconference - Volume II
A hybrid deliberative layer for robotic agents: fusing DL reasoning with HTN planning in autonomous robots
Automating the evaluation of planning systems
AI Communications
Hi-index | 0.00 |
Recent trends in planning research have led to empirical comparison becoming commonplace. The field has started to settle into a methodology for such comparisons, which for obvious practical reasons requires running a subset of planners on a subset of problems. In this paper, we characterize the methodology and examine eight implicit assumptions about the problems, planners and metrics used in many of these comparisons. The problem assumptions are: PR1) the performance of a general purpose planner should not be penalized/biased if executed on a sampling of problems and domains, PR2) minor syntactic differences in representation do not affect performance, and PR3) problems should be solvable by STRIPS capable planners unless they require ADL. The planner assumptions are: PL1) the latest version of a planner is the best one to use, PL2) default parameter settings approximate good performance, and PL3) time cut-offs do not unduly bias outcome. The metrics assumptions are: M1) performance degrades similarly for each planner when run on degraded runtime environments (e.g., machine platform) and M2) the number of plan steps distinguishes performance. We find that most of these assumptions are not supported empirically; in particular, that planners are affected differently by these assumptions. We conclude with a call to the community to devote research resources to improving the state of the practice and especially to enhancing the available benchmark problems.