Bayeux: an architecture for scalable and fault-tolerant wide-area data dissemination
NOSSDAV '01 Proceedings of the 11th international workshop on Network and operating systems support for digital audio and video
Design and evaluation of a wide-area event notification service
ACM Transactions on Computer Systems (TOCS)
Chord: A scalable peer-to-peer lookup service for internet applications
Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications
The JEDI Event-Based Infrastructure and Its Application to the Development of the OPSS WFMS
IEEE Transactions on Software Engineering
An Efficient k-Means Clustering Algorithm: Analysis and Implementation
IEEE Transactions on Pattern Analysis and Machine Intelligence
Application-Level Multicast Using Content-Addressable Networks
NGC '01 Proceedings of the Third International COST264 Workshop on Networked Group Communication
Clustering Algorithms for Content-Based Publication-Subscription Systems
ICDCS '02 Proceedings of the 22 nd International Conference on Distributed Computing Systems (ICDCS'02)
A method for decentralized clustering in large multi-agent systems
AAMAS '03 Proceedings of the second international joint conference on Autonomous agents and multiagent systems
YFilter: Efficient and Scalable Filtering of XML Documents
ICDE '02 Proceedings of the 18th International Conference on Data Engineering
Tapestry: An Infrastructure for Fault-tolerant Wide-area Location and
Tapestry: An Infrastructure for Fault-tolerant Wide-area Location and
HYPER: A Hybrid Approach to Efficient Content-Based Publish/Subscribe
ICDCS '05 Proceedings of the 25th IEEE International Conference on Distributed Computing Systems
CoNEXT '05 Proceedings of the 2005 ACM conference on Emerging network experiment and technology
On the database/network interface in large-scale publish/subscribe systems
Proceedings of the 2006 ACM SIGMOD international conference on Management of data
Fast and exact out-of-core and distributed k-means clustering
Knowledge and Information Systems
Corona: a high performance publish-subscribe system for the world wide web
NSDI'06 Proceedings of the 3rd conference on Networked Systems Design & Implementation - Volume 3
Towards an internet-scale XML dissemination service
VLDB '04 Proceedings of the Thirtieth international conference on Very large data bases - Volume 30
GREEN: a configurable and re-configurable publish-subscribe middleware for pervasive computing
OTM'05 Proceedings of the 2005 Confederated international conference on On the Move to Meaningful Internet Systems - Volume >Part I
FeedTree: sharing web micronews with peer-to-peer event notification
IPTPS'05 Proceedings of the 4th international conference on Peer-to-Peer Systems
Scribe: a large-scale and decentralized application-level multicast infrastructure
IEEE Journal on Selected Areas in Communications
Early profile pruning on XML-aware publish-subscribe systems
VLDB '07 Proceedings of the 33rd international conference on Very large data bases
Large scale P2P distribution of open-source software
VLDB '07 Proceedings of the 33rd international conference on Very large data bases
Enriching topic-based publish-subscribe systems with related content
Proceedings of the 2008 ACM SIGMOD international conference on Management of data
Value-based predicate filtering of XML documents
Data & Knowledge Engineering
Optimized union of non-disjoint distributed data sets
Proceedings of the 12th International Conference on Extending Database Technology: Advances in Database Technology
Fast XML document filtering by sequencing twig patterns
ACM Transactions on Internet Technology (TOIT)
Reliable publish/subscribe middleware for time-sensitive internet-scale applications
Proceedings of the Third ACM International Conference on Distributed Event-Based Systems
Publish-subscribe services for urgent and emergency response
Proceedings of the Fourth International ICST Conference on COMmunication System softWAre and middlewaRE
Ripple: A publish/subscribe service for multidata item updates propagation in the cloud
Journal of Network and Computer Applications
RoSeS: a continuous content-based query engine for RSS feeds
DEXA'11 Proceedings of the 22nd international conference on Database and expert systems applications - Volume Part II
DRScribe: an improved topic-based publish-subscribe system with dynamic routing
WAIM'11 Proceedings of the 12th international conference on Web-age information management
STAIRS: Towards efficient full-text filtering and dissemination in DHT environments
The VLDB Journal — The International Journal on Very Large Data Bases
Distributed top-k full-text content dissemination
Distributed and Parallel Databases
| Hi-index | 0.01 |
We consider in this paper a class of Publish-Subscribe (pub-sub) systems called topic-based systems, where users subscribe to topics and are notified on events that belong to those subscribed topics. With the recent flourishing of RSS news syndication, these systems are regaining popularity and are raising new challenging problems. In most of the modern topics-based systems, the events in each topic are delivered to the subscribers via a supporting, distributed, data structure (typically a multicast tree). Since peers in the network may come and go frequently, this supporting structure must be continuously maintained so that "holes" do not disrupt the events delivery. The dissemination of events in each topic thus incurs two main costs: (1) the actual transmission cost for the topic events,and (2) the maintenance cost for its supporting structure. This maintenance overhead becomes particularly dominating when a pub-sub system supports a large number of topics with moderate event frequency; a typical scenario in nowadays news syndication scene. The goal of this paper is to devise a method for reducing this maintenance overhead to the minimum. Our aim is not to invent yet another topic-based pub-sub system, but rather to develop a generic technique for better utilization of existing platforms. Our solution is based on a novel distributed clustering algorithm that utilizes correlations between user subscriptions to dynamically group topics together, into virtual topics (called topic-clusters), andt hereby unifies their supporting structures and reduces costs. Our technique continuously adapts the topic-clusters and the user subscriptions to the system state, and incurs only very minimal overhead. We have implemented our solution in the Tamara pub-sub system. Our experimental study shows this approach to be extremely effective, improving the performance by an order of magnitude.