Sampling, information extraction and summarisation of hidden web databases

  • Authors:
  • Yih-Ling Hedley;Muhammad Younas;Anne James;Mark Sanderson

  • Affiliations:
  • School of Mathematical and Information Sciences, Coventry University, Coventry, United Kingdom;Department of Computing, Oxford Brookes University, Oxford, United Kingdom;School of Mathematical and Information Sciences, Coventry University, Coventry, United Kingdom;Department of Information Studies, University of Sheffield, Sheffield, United Kingdom

  • Venue:
  • Data & Knowledge Engineering - Special issue: WIDM 2004
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

Hidden Web databases maintain a collection of specialised documents, which are dynamically generated using page templates. This paper presents the Two-Phase Sampling (2PS) technique that detects and extracts query-related information from documents contained in databases. 2PS is based on a two-phase framework for the sampling, information extraction and summarisation of Hidden Web documents. In the first phase, 2PS samples and stores documents for further analysis. In the second phase, it detects Web page templates from sampled documents and extracts relevant information from which a content summary is then generated. Experimental results demonstrate that 2PS effectively eliminates irrelevant information from sampled documents and generates terms and frequencies with improved accuracy.