Mission-phasing techniques for constrained agents in stochastic environments

• Authors: Edmund H. Durfee;Jianhui Wu
• Venue: Mission-phasing techniques for constrained agents in stochastic environments
• Year: 2008

Resource constraints restrict the set of actions that an agent can take, such that the agent might not be able to perform all its desired tasks. Computational time limitations restrict the number of states that an agent can model and reason over, such that the agent might not be able to formulate a policy that can respond to all possible eventualities. This work argues that, in either situation, one effective way of improving the agent's performance is to adopt a phasing strategy. Resource-constrained agents can choose to reconfigure resources and switch action sets for handling upcoming events better when moving from phase to phase; time-limited agents can choose to focus computation on high-value phases and to exploit additional computation time during the execution of earlier phases to improve solutions for future phases. This dissertation consists of two parts, corresponding to the aforementioned resource constraints and computational time limitations. The first part of the dissertation focuses on the development of automated resource-driven mission-phasing techniques for agents operating in resource-constrained environments. We designed a suite of algorithms which not only can find solutions to optimize the use of predefined phase-switching points, but can also automatically determine where to establish such points, accounting for the cost of creating them, in complex stochastic environments. By formulating the coupled problems of mission decomposition, resource configuration, and policy formulation into a single compact mathematical formulation, the presented algorithms can effectively exploit problem structure and often considerably reduce computational cost for finding exact solutions. The second part of this dissertation is the design of computation-driven mission-phasing techniques for time-critical systems. We developed a new deliberation scheduling approach, which can simultaneously solve the coupled problems of deciding both when to deliberate given its cost, and which phase decision procedures to execute during deliberation intervals. Meanwhile, we designed a heuristic search method to effectively utilize the allocated time within each phase. As illustrated in analytical and experimental results, the computation-driven mission-phasing techniques, which extend problem decomposition techniques with the across-phase deliberation scheduling and inner-phase heuristic search methods mentioned above, can help an agent judiciously emphasize high-value portions of a large problem, while paying less attention to others, to generate a better policy within its time limit.