XPathLearner: an on-line self-tuning Markov histogram for XML path selectivity estimation

  • Authors:

  • Lipyeow Lim;Min Wang;Sriram Padmanabhan;Jeffrey Scott Vitter;Ronald Parr

  • Affiliations:

  • Department of Computer Science, Duke University, Durham, NC;IBM T. J. Watson Research Center, Hawthorne NY;IBM T. J. Watson Research Center, Hawthorne NY;Department of Computer Science, Duke University, Durham, NC;Department of Computer Science, Duke University, Durham, NC

  • Venue:
  • VLDB '02 Proceedings of the 28th international conference on Very Large Data Bases
  • Year:
  • 2002

Quantified Score

Hi-index 0.00

Visualization

Abstract

The extensible mark-up language (XML) is gaining widespread use as a format for data exchange and storage on the World Wide Web. Queries over XML data require accurate selectivity estimation of path expressions to optimize query execution plans. Selectivity estimation of XML path expression is usually done based on summary statistics about the structure of the underlying XML repository. All previous methods require an off-line scan of the XML repository to collect the statistics. In this paper, we propose XPathLearner, a method for estimating selectivity of the most commonly used types of path expressions without looking at the XML data. XPathLearner gathers and refines the statistics using query feedback in an on-line manner and is especially suited to queries in Internet scale applications since the underlying XML repository is either inaccessible or too large to be scanned in its entirety. Besides the on-line property, our method also has two other novel features: (a) XPathLearner is workload-aware in collecting the statistics and thus can be more accurate than the more costly off-line method under tight memory constraints, and (b) XPathLearner automatically adjusts the statistics using query feedback when the underlying XML data change. We show empirically the estimation accuracy of our method using several real data sets.