Web-page classification through summarization

  • Authors:

  • Dou Shen;Zheng Chen;Qiang Yang;Hua-Jun Zeng;Benyu Zhang;Yuchang Lu;Wei-Ying Ma

  • Affiliations:

  • Tsinghua University, Beijing, P.R. China;Microsoft Research Asia, Beijing, P.R. China;Hong Kong University of Science and Technology, Kowloon, Hong Kong;Microsoft Research Asia, Beijing, P.R. China;Microsoft Research Asia, Beijing, P.R. China;Tsinghua University, Beijing, P.R. China;Microsoft Research Asia, Beijing, P.R. China

  • Venue:
  • Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

Web-page classification is much more difficult than pure-text classification due to a large variety of noisy information embedded in Web pages. In this paper, we propose a new Web-page classification algorithm based on Web summarization for improving the accuracy. We first give empirical evidence that ideal Web-page summaries generated by human editors can indeed improve the performance of Web-page classification algorithms. We then propose a new Web summarization-based classification algorithm and evaluate it along with several other state-of-the-art text summarization algorithms on the LookSmart Web directory. Experimental results show that our proposed summarization-based classification algorithm achieves an approximately 8.8% improvement as compared to pure-text-based classification algorithm. We further introduce an ensemble classifier using the improved summarization algorithm and show that it achieves about 12.9% improvement over pure-text based methods.