Leveraging distributed publish/subscribe systems for scalable stream query processing

Authors:
Yongluan Zhou;Kian-Lee Tan;Feng Yu
Affiliations:
National University of Singapore;National University of Singapore;National University of Singapore
Venue:
BIRTE'06 Proceedings of the 1st international conference on Business intelligence for the real-time enterprises
Year:
2006

Citing 12
Cited 1

The R*-tree: an efficient and robust access method for points and rectangles

SIGMOD '90 Proceedings of the 1990 ACM SIGMOD international conference on Management of data
Parallel dynamic graph partitioning for adaptive unstructured meshes

Journal of Parallel and Distributed Computing - Special issue on dynamic load balancing
Multilevel k-way partitioning scheme for irregular graphs

Journal of Parallel and Distributed Computing
Matching events in a content-based subscription system

Proceedings of the eighteenth annual ACM symposium on Principles of distributed computing
Design and evaluation of a wide-area event notification service

ACM Transactions on Computer Systems (TOCS)
Graph partitioning for high-performance scientific simulations

Sourcebook of parallel computing
SemCast: Semantic Multicast for Content-Based Data Dissemination

ICDE '05 Proceedings of the 21st International Conference on Data Engineering
Dynamic Load Distribution in the Borealis Stream Processor

ICDE '05 Proceedings of the 21st International Conference on Data Engineering
Operator placement for in-network stream query processing

Proceedings of the twenty-fourth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Network-Aware Operator Placement for Stream-Processing Systems

ICDE '06 Proceedings of the 22nd International Conference on Data Engineering
Monitoring streams: a new class of data management applications

VLDB '02 Proceedings of the 28th international conference on Very Large Data Bases
Network-aware query processing for stream-based applications

VLDB '04 Proceedings of the Thirtieth international conference on Very large data bases - Volume 30

Top-k/w publish/subscribe: A publish/subscribe model for continuous top-k processing over data streams

Information Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

Existing distributed publish/subscribe systems (DPSS) offer loosely coupled and easy to deploy content-based stream delivery services to a large number of users. However, the lack of query expressiveness limits their application scope. On the other hand, distributed stream processing engines (DSPE) provide efficient processing services for complex stream queries. Nevertheless, these systems are typically tightly coupled, platform dependent, difficult to deploy and maintain, and less scalable to the number of users. In this paper, we propose a new architectural design for a scalable distributed stream processing system, which provides services to evaluate continuous queries for a large number of clients. It is built by placing a query layer on top of a DPSS architecture. In particular, we focus on solving the query distribution problem in the query layer.