The R*-tree: an efficient and robust access method for points and rectangles
SIGMOD '90 Proceedings of the 1990 ACM SIGMOD international conference on Management of data
Matching events in a content-based subscription system
Proceedings of the eighteenth annual ACM symposium on Principles of distributed computing
Filtering algorithms and implementation for very fast publish/subscribe systems
SIGMOD '01 Proceedings of the 2001 ACM SIGMOD international conference on Management of data
Design and evaluation of a wide-area event notification service
ACM Transactions on Computer Systems (TOCS)
Efficient Filtering of XML Documents for Selective Dissemination of Information
VLDB '00 Proceedings of the 26th International Conference on Very Large Data Bases
The many faces of publish/subscribe
ACM Computing Surveys (CSUR)
Forwarding in a content-based network
Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications
Meghdoot: content-based publish/subscribe over P2P networks
Proceedings of the 5th ACM/IFIP/USENIX international conference on Middleware
Analysis and Algorithms for Content-Based Event Matching
ICDCSW '05 Proceedings of the Fourth International Workshop on Distributed Event-Based Systems (DEBS) (ICDCSW'05) - Volume 04
MapReduce: simplified data processing on large clusters
OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
The arbitrary Boolean publish/subscribe model: making the case
Proceedings of the 2007 inaugural international conference on Distributed event-based systems
Bloom filter based routing for content-based publish/subscribe
Proceedings of the second international conference on Distributed event-based systems
Scalable Distribution of XML Content with XNet
IEEE Transactions on Parallel and Distributed Systems
Parallel event processing for content-based publish/subscribe systems
Proceedings of the Third ACM International Conference on Distributed Event-Based Systems
Template Matching Techniques in Computer Vision: Theory and Practice
Template Matching Techniques in Computer Vision: Theory and Practice
Efficiently evaluating complex boolean expressions
Proceedings of the 2010 ACM SIGMOD International Conference on Management of data
Publisher Placement Algorithms in Content-Based Publish/Subscribe
ICDCS '10 Proceedings of the 2010 IEEE 30th International Conference on Distributed Computing Systems
Proceedings of the 17th ACM conference on Computer and communications security
Load Balancing Content-Based Publish/Subscribe Systems
ACM Transactions on Computer Systems (TOCS)
DEXA'10 Proceedings of the 21st international conference on Database and expert systems applications: Part I
S4: Distributed Stream Computing Platform
ICDMW '10 Proceedings of the 2010 IEEE International Conference on Data Mining Workshops
Towards Approximate Event Processing in a Large-Scale Content-Based Network
ICDCS '11 Proceedings of the 2011 31st International Conference on Distributed Computing Systems
Foundations for Highly Available Content-Based Publish/Subscribe Overlays
ICDCS '11 Proceedings of the 2011 31st International Conference on Distributed Computing Systems
Green Resource Allocation Algorithms for Publish/Subscribe Systems
ICDCS '11 Proceedings of the 2011 31st International Conference on Distributed Computing Systems
Split and Subsume: Subscription Normalization for Effective Content-Based Messaging
ICDCS '11 Proceedings of the 2011 31st International Conference on Distributed Computing Systems
A Novel Approach to QoS Monitoring in the Cloud
CCP '11 Proceedings of the 2011 First International Conference on Data Compression, Communications and Processing
Scalable and Low-Latency Data Processing with Stream MapReduce
CLOUDCOM '11 Proceedings of the 2011 IEEE Third International Conference on Cloud Computing Technology and Science
C-MR: continuously executing MapReduce workflows on multi-core processors
Proceedings of third international workshop on MapReduce and its Applications Date
Thrifty privacy: efficient support for privacy-preserving publish/subscribe
Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems
Community Clustering for Distributed Publish/Subscribe Systems
CLUSTER '12 Proceedings of the 2012 IEEE International Conference on Cluster Computing
Opportunistic multipath forwarding in content-based publish/subscribe overlays
Proceedings of the 13th International Middleware Conference
| Hi-index | 0.00 |
By routing messages based on their content, publish/subscribe (pub/sub) systems remove the need to establish and maintain fixed communication channels. Pub/sub is a natural candidate for designing large-scale systems, composed of applications running in different domains and communicating via middleware solutions deployed on a public cloud. Such pub/sub systems must provide high throughput, filtering thousands of publications per second matched against hundreds of thousands of registered subscriptions with low and predictable delays, and must scale horizontally and vertically. As large-scale application composition may require complex publications and subscriptions representations, pub/sub system designs should not rely on the specific characteristics of a particular filtering scheme for implementing scalability. In this paper, we depart from the use of broker overlays, where each server must support the whole range of operations of a pub/sub service, as well as overlay management and routing functionality. We propose instead a novel and pragmatic tiered approach to obtain high-throughput and scalable pub/sub for clusters and cloud deployments. We separate the three operations involved in pub/sub and leverage their natural potential for parallelization. Our design, named StreamHub, is oblivious to the semantics of subscriptions and publications. It can support any type and number of filtering operations implemented by independent libraries. Experiments on a cluster with up to 384 cores indicate that StreamHub is able to register 150 K subscriptions per second and filter next to 2 K publications against 100 K stored subscriptions, resulting in nearly 400 K notifications sent per second. Comparisons against a broker overlay solution shows an improvement of two orders of magnitude in throughput when using the same number of cores.