Implementing a scalable XML publish/subscribe system using relational database systems

Authors:
Feng Tian;Berthold Reinwald;Hamid Pirahesh;Tobias Mayr;Jussi Myllymaki
Affiliations:
University of Wisconsin, Madison, Madison, WI;IBM Almaden Research Center, San Jose, CA;IBM Almaden Research Center, San Jose, CA;IBM Almaden Research Center, San Jose, CA;IBM Almaden Research Center, San Jose, CA
Venue:
SIGMOD '04 Proceedings of the 2004 ACM SIGMOD international conference on Management of data
Year:
2004

Citing 15
Cited 30

Multiple-query optimization

ACM Transactions on Database Systems (TODS)
The process group approach to reliable distributed computing

Communications of the ACM
The Information Bus: an architecture for extensible distributed systems

SOSP '93 Proceedings of the fourteenth ACM symposium on Operating systems principles
Group communication

Communications of the ACM
Matching events in a content-based subscription system

Proceedings of the eighteenth annual ACM symposium on Principles of distributed computing
NiagaraCQ: a scalable continuous query system for Internet databases

SIGMOD '00 Proceedings of the 2000 ACM SIGMOD international conference on Management of data
A Performance Comparison of the Rete and TREAT Algorithms for Testing Database Rule Conditions

Proceedings of the Eighth International Conference on Data Engineering
Processing XML Streams with Deterministic Automata

ICDT '03 Proceedings of the 9th International Conference on Database Theory
Efficient Filtering of XML Documents for Selective Dissemination of Information

VLDB '00 Proceedings of the 26th International Conference on Very Large Data Bases
Scalable Trigger Processing

ICDE '99 Proceedings of the 15th International Conference on Data Engineering
Stream processing of XPath queries with predicates

Proceedings of the 2003 ACM SIGMOD international conference on Management of data
XPath queries on streaming data

Proceedings of the 2003 ACM SIGMOD international conference on Management of data
YFilter: Efficient and Scalable Filtering of XML Documents

ICDE '02 Proceedings of the 18th International Conference on Data Engineering
Efficient Filtering of XML Documents with XPath Expressions

ICDE '02 Proceedings of the 18th International Conference on Data Engineering
Design and Evaluation of Alternative Selection Placement Strategies in Optimizing Continuous Queries

ICDE '02 Proceedings of the 18th International Conference on Data Engineering

Scalable security and accounting services for content-based publish/subscribe systems

Proceedings of the 2005 ACM symposium on Applied computing
FiST: scalable XML document filtering by sequencing twig patterns

VLDB '05 Proceedings of the 31st international conference on Very large data bases
AFilter: adaptable XML filtering with prefix-caching suffix-clustering

VLDB '06 Proceedings of the 32nd international conference on Very large data bases
Triggers over nested views of relational data

ACM Transactions on Database Systems (TODS)
An efficient subscription routing algorithm for scalable XML-based publish/subscribe systems

Journal of Systems and Software
Towards an internet-scale XML dissemination service

VLDB '04 Proceedings of the Thirtieth international conference on Very large data bases - Volume 30
On adopting Content-Based Routing in service-oriented architectures

Information and Software Technology
Early profile pruning on XML-aware publish-subscribe systems

VLDB '07 Proceedings of the 33rd international conference on Very large data bases
XFIS: an XML filtering system based on string representation and matching

International Journal of Web Engineering and Technology
Efficient processing of branch queries for high-performance XML filtering

Proceedings of the 2nd international conference on Scalable information systems
Distributed XML processing: Theory and applications

Journal of Parallel and Distributed Computing
A scalable and self-adapting notification framework for healthcare information systems

Proceedings of the 1st international conference on PErvasive Technologies Related to Assistive Environments
Value-based predicate filtering of XML documents

Data & Knowledge Engineering
The Å Publish/Subscribe Framework

UIC '09 Proceedings of the 6th International Conference on Ubiquitous Intelligence and Computing
Fast XML document filtering by sequencing twig patterns

ACM Transactions on Internet Technology (TOIT)
Publish-subscribe services for urgent and emergency response

Proceedings of the Fourth International ICST Conference on COMmunication System softWAre and middlewaRE
Web service decomposition: Edge computing architecture for cache-friendly e-commerce applications

Electronic Commerce Research and Applications
An XML publish/subscribe algorithm implemented by relational operators

APWeb/WAIM'07 Proceedings of the joint 9th Asia-Pacific web and 8th international conference on web-age information management conference on Advances in data and web management
Parallelizing XML data-streaming workflows via MapReduce

Journal of Computer and System Sciences
XMin: Minimizing Tree Pattern Queries with Minimality Guarantee

World Wide Web
Distributed structural and value XML filtering

Proceedings of the Fourth ACM International Conference on Distributed Event-Based Systems
Efficient filtering of XML documents with XPath expressions containing ancestor axis

WAIM'10 Proceedings of the 11th international conference on Web-age information management
A scalable and self-adapting notification framework

DEXA'10 Proceedings of the 21st international conference on Database and expert systems applications: Part II
Fine-grained parallel XML filtering for content-based publish/subscribe systems

Proceedings of the 5th ACM international conference on Distributed event-based system
Relational approach for shortest path discovery over large graphs

Proceedings of the VLDB Endowment
Collection-Oriented scientific workflows for integrating and analyzing biological data

DILS'06 Proceedings of the Third international conference on Data Integration in the Life Sciences
Evolution and reactivity for the web

Proceedings of the First international conference on Reasoning Web
Data management support for notification services

Data Management in a Connected World
XML filtering with XPath expressions containing parent and ancestor axes

Information Sciences: an International Journal
FoXtrot: Distributed structural and value XML filtering

ACM Transactions on the Web (TWEB)

Quantified Score

Hi-index	0.00

Visualization

Abstract

An XML publish/subscribe system needs to match many XPath queries (subscriptions) over published XML documents. The performance and scalability of the matching algorithm is essential for the system when the number of XPath subscriptions is large. Earlier solutions to this problem usually built large finite state automata for all the XPath subscriptions in memory. The scalability of this approach is limited by the amount of available physical memory. In this paper, we propose an implementation that uses a relational database as the matching engine. The heavy lifting part of evaluating a large number of subscriptions is done inside a relational database using indices and joins. We described several different implementation strategies and presented a performance evaluation. The system shows very good performance and scalability in our experiments, handling millions of subscriptions with moderate amount of physical memory.