Distributed community crawling

Authors:
Fabrizio Costa;Paolo Frasconi
Affiliations:
Università degli Studi di Firenze, Italy;Università degli Studi di Firenze, Italy
Venue:
Proceedings of the 13th international World Wide Web conference on Alternate track papers & posters
Year:
2004

Citing 4
Cited 0

Silk from a sow's ear: extracting usable structures from the Web

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Trawling the Web for emerging cyber-communities

WWW '99 Proceedings of the eighth international conference on World Wide Web
Breadth-first crawling yields high-quality pages

Proceedings of the 10th international conference on World Wide Web
Parallel crawlers

Proceedings of the 11th international conference on World Wide Web

Quantified Score

Hi-index	0.00

Visualization

Abstract

The massive distribution of the crawling task can lead to inefficient exploration of the same portion of the Web. We propose a technique to guide crawlers exploration based on the notion of Web communities. Thest ability properties of the method can be used as an implicit coordination mechanism to increase the efficiency of the crawling task.