Restrictive clustering and metaclustering for self-organizing document collections
Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
The SphereSearch engine for unified ranked retrieval of heterogeneous XML and web documents
VLDB '05 Proceedings of the 31st international conference on Very large data bases
VLDB '03 Proceedings of the 29th international conference on Very large data bases - Volume 29
| Hi-index | 0.00 |
Focused (thematic) crawling is a relatively new, promising approach to improving the recall of expert search on the Web.It involves the automatic classification of visited documents into a user- or community-specific topic hierarchy (ontology).The quality of the training data for the classifier is the most critical issue and potential bottleneck for the effectivity and scale of a focused crawler.This paper presents the BINGO! approach to focused crawling that aims to overcome the limitations of the initial training data.To this end, BINGO! identifies, among the crawled and positively classified documents of a topic, characteristic "archetypes" and uses them for periodically re-training the classifier; this way the crawler is dynamically adapted based on the most significant documents seen so far.Two kinds of archetypes are considered;good authorities as determined by employing Klienberg's link analysis algorithm, and documents that have been automatically classified with high confidence using a linear SVM classifier.Our approach is fully implemented in the BINGO! system, and our experiments indicate that the dynamic enhancement of training data based on archetypes extends the "knowledge base" of the classifier by a substantial margin without loss of classification accuracy.