Two-phase Web site classification based on Hidden Markov Tree models

  • Authors:

  • Yong-Hong Tian;Tie-Jun Huang;Wen Gao

  • Affiliations:

  • Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100080, China;Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100080, China and Graduate School of Chinese Academy of Sciences, Beijing 100039, China;Inst. of Comp. Technol., Chinese Acad. of Sci., Beijing 100080, China and Grad. Sch. of Ch. Acad. of Sci., Beijing 100039, China and Dept. of Comp. Sci., Harbin Inst. of Technol., Harbin 150001, C ...

  • Venue:
  • Web Intelligence and Agent Systems
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

The extensive amount of diversified Web-based information necessitates the development of automated subject-specific Web site classification techniques. Given that Web sites are in essence heterogeneous, multi-structured and often accompanied with much noise, it is important to design Web site classification algorithms that can scale well in the context of noise and heterogeneity. In this paper, we propose a novel approach for Web site classification based on the content, structure and context information of Web sites. In our approach, the site structure is represented as a two-layered tree, i.e., each page is modeled as a DOM (Document Object Model) tree, and a page tree is used to hierarchically link all pages within the site. Two context models are formulated to characterize the topical dependences between nodes in the two-layered tree. Using the Hidden Markov Tree (HMT) as the statistical model of page trees and DOM trees, a two-phase Web site classification algorithm is presented. Moreover, for further improving accuracy while reducing the classification overheads, a two-stage denoising procedure is adopted to remove the noise information within sites, and an entropy-based strategy is introduced to dynamically prune the page trees. The experiments demonstrate that the proposed approach is able to offer high accuracy and efficient processing performance.