A new method for focused crawler cross tunnel

  • Authors:

  • Na Luo;Wanli Zuo;Fuyu Yuan;Changli Zhang

  • Affiliations:

  • College of Computer Science and Technology, JiLin University Key Laboratory of, Symbol Computation and Knowledge Engineering of the Ministry of Education, ChangChun, P.R. China;College of Computer Science and Technology, JiLin University Key Laboratory of, Symbol Computation and Knowledge Engineering of the Ministry of Education, ChangChun, P.R. China;College of Computer Science and Technology, JiLin University Key Laboratory of, Symbol Computation and Knowledge Engineering of the Ministry of Education, ChangChun, P.R. China;College of Computer Science and Technology, JiLin University Key Laboratory of, Symbol Computation and Knowledge Engineering of the Ministry of Education, ChangChun, P.R. China

  • Venue:
  • RSKT'06 Proceedings of the First international conference on Rough Sets and Knowledge Technology
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

Focused crawlers are programs designed to selectively retrieve Web pages relevant to a specific domain for the use of domain-specific search engines. Tunneling is a heuristic-based method that solves global optimization problem. In this paper we use content block algorithm to enhance focused crawler's ability of traversing tunnel. The novel Algorithm not only avoid granularity too coarse when evaluation on the whole page but also avoid granularity too fine based on link-context. A comprehensive experiment has been conducted, the result shows obviously that this approach outperforms BestFirst and Anchor text algorithm both in harvest ratio and efficiency