A distributed full-text top-k document dissemination system in distributed hash tables

  • Authors:
  • Weixiong Rao;Lei Chen

  • Affiliations:
  • Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Kowloon, China and School of Computer and Information Engineering, HeNan University, Kaifeng, Ch ...;Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Kowloon, China and School of Computer and Information Engineering, HeNan University, Kaifeng, Ch ...

  • Venue:
  • World Wide Web
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Recent years witnessed the explosive growth of `live' web content in the World Wide Web like Weblogs, RSS feeds, and real-time news, etc. The popular usage of RSS feeds/readers enables end users to subscribe for favorite contents via input RSS URLs. However, the RSS feeds/readers architecture suffers from (i) the high bandwidth consumption issue, and (ii) limited filtering semantics. In this paper, we proposed a stateful full text dissemination scheme over structured P2Ps to address both issues. Specifically, for the semantic side, end users are allowed to subscribe for favorite contents via input keywords; for the network bandwidth side, the cooperative content polling, filtering and disseminating via DHT-based P2P overlay networks save the network bandwidth consumption. Our contributions include the novel techniques to (i) reduce the unit-publishing cost by pruning irreverent documents during the forwarding path towards destinations, and (ii) reduce the publication amount by selecting a very small number of meaningful terms. Based on real data sets, our experimental results show that the proposed scheme can significantly reduce the publishing cost with low maintenance overhead and a high document quality.