Speculative plan execution for information agents

• Authors: Craig A. Knoblock;Greg Barish
• Venue: Speculative plan execution for information agents
• Year: 2003

While information agents make it possible to gather, combine, and process data on networks like the Internet, execution performance often suffers due to remote source latencies. Agents do not control remote sources and must wait an undetermined amount of time for a query to be answered. The problem becomes worse when an agent plan requires that the answers provided by one source be used as a basis for querying another source. In this dissertation, I make three related contributions that address these problems and significantly improve information agent performance. The first is an expressive agent plan language and a streaming dataflow execution system. The combination of both allows agent plans to be described and efficiently executed, realizing the maximum parallelism allowable by the data dependencies in the plan. My experimental results confirm that execution is efficient and that the plan language is expressive enough to support tasks beyond those supported by traditional network query engines, such as recursive information gathering and monitoring. A second contribution is a strategy for speculative plan execution within a streaming dataflow architecture. Under speculative execution, certain operators are issued ahead of schedule, using data predicted from experience. Through this technique, remaining costly data dependencies between I/O-bound operators can be broken, leading to parallelism beyond the normal dataflow limit. My experimental results demonstrate that speculative execution can lead to significant speedups in both Web agent plans as well as certain types of queries for distributed database systems. A third contribution is a technique for learning how to predict data for speculative plan execution. This approach combines caching with classification and transduction as a means for predicting future values from prior hints. Classification and transduction are more space efficient than caching and can improve the accuracy of prediction because each is capable of responding to new hints. The resulting improved accuracy increases the utility of speculative execution and leads to greater average plan speedups. My experimental results for a set of Web agent plans confirm these space-efficiency and accuracy claims.