Calendar queues: a fast 0(1) priority queue implementation for the simulation event set problem
Communications of the ACM
Continuous queries over append-only databases
SIGMOD '92 Proceedings of the 1992 ACM SIGMOD international conference on Management of data
Wide area traffic: the failure of Poisson modeling
IEEE/ACM Transactions on Networking (TON)
Exploiting Punctuation Semantics in Continuous Data Streams
IEEE Transactions on Knowledge and Data Engineering
Aurora: a new model and architecture for data stream management
The VLDB Journal — The International Journal on Very Large Data Bases
Characterizing memory requirements for queries over continuous data streams
ACM Transactions on Database Systems (TODS)
Nile: A Query Processing Engine for Data Streams
ICDE '04 Proceedings of the 20th International Conference on Data Engineering
Adaptive ordering of pipelined stream filters
SIGMOD '04 Proceedings of the 2004 ACM SIGMOD international conference on Management of data
Static optimization of conjunctive queries with sliding windows over infinite streams
SIGMOD '04 Proceedings of the 2004 ACM SIGMOD international conference on Management of data
Exploiting k-constraints to reduce memory overhead in continuous queries over data streams
ACM Transactions on Database Systems (TODS)
Operator scheduling in data stream systems
The VLDB Journal — The International Journal on Very Large Data Bases
Flexible time management in data stream systems
PODS '04 Proceedings of the twenty-third ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
The CQL continuous query language: semantic foundations and query execution
The VLDB Journal — The International Journal on Very Large Data Bases
Maximizing the output rate of multi-way join queries over streaming information sources
VLDB '03 Proceedings of the 29th international conference on Very large data bases - Volume 29
Processing sliding window multi-joins in continuous queries over data streams
VLDB '03 Proceedings of the 29th international conference on Very large data bases - Volume 29
Operator scheduling in a data stream manager
VLDB '03 Proceedings of the 29th international conference on Very large data bases - Volume 29
Query languages and data models for database sequences and data streams
VLDB '04 Proceedings of the Thirtieth international conference on Very large data bases - Volume 30
On-the-fly sharing for streamed aggregation
Proceedings of the 2006 ACM SIGMOD international conference on Management of data
A data stream language and system designed for power and extensibility
CIKM '06 Proceedings of the 15th ACM international conference on Information and knowledge management
Incremental Evaluation of Sliding-Window Queries over Data Streams
IEEE Transactions on Knowledge and Data Engineering
Replay-based approaches to revision processing in stream query engines
SSPS '08 Proceedings of the 2nd international workshop on Scalable stream processing system
Semantics and implementation of continuous sliding window queries over data streams
ACM Transactions on Database Systems (TODS)
Synergy: sharing-aware component composition for distributed stream processing systems
Proceedings of the ACM/IFIP/USENIX 2006 International Conference on Middleware
Flexible and scalable storage management for data-intensive stream processing
Proceedings of the 12th International Conference on Extending Database Technology: Advances in Database Technology
Mode Aware Stream Query Processing
SSDBM 2009 Proceedings of the 21st International Conference on Scientific and Statistical Database Management
Window Update Patterns in Stream Operators
ADBIS '09 Proceedings of the 13th East European Conference on Advances in Databases and Information Systems
A simple but effective event-driven model for data stream queries
APWeb/WAIM'07 Proceedings of the joint 9th Asia-Pacific web and 8th international conference on web-age information management conference on Advances in data and web management
Relational languages and data models for continuous queries on sequences and data streams
ACM Transactions on Database Systems (TODS)
Update propagation in a streaming warehouse
SSDBM'11 Proceedings of the 23rd international conference on Scientific and statistical database management
Subsuming multiple sliding windows for shared stream computation
ADBIS'11 Proceedings of the 15th international conference on Advances in databases and information systems
Synergy: sharing-aware component composition for distributed stream processing systems
Middleware'06 Proceedings of the 7th ACM/IFIP/USENIX international conference on Middleware
On concurrency control in sliding window queries over data streams
EDBT'06 Proceedings of the 10th international conference on Advances in Database Technology
Window specification over data streams
EDBT'06 Proceedings of the 2006 international conference on Current Trends in Database Technology
Processing flows of information: From data stream to complex event processing
ACM Computing Surveys (CSUR)
Revisiting formal ordering in data stream querying
Proceedings of the 27th Annual ACM Symposium on Applied Computing
Transactional stream processing
Proceedings of the 15th International Conference on Extending Database Technology
Mining frequent itemsets over tuple-evolving data streams
Proceedings of the 28th Annual ACM Symposium on Applied Computing
Hi-index | 0.00 |
A defining characteristic of continuous queries over on-line data streams, possibly bounded by sliding windows, is the potentially infinite and time-evolving nature of their inputs and outputs. New items continually arrive on the input streams and new results are continually produced. Additionally, inputs expire by falling out of range of their sliding windows and results expire when they cease to satisfy the query. This impacts continuous query processing in two ways. First, data stream systems allow tables to be queried alongside data streams, but in terms of query semantics, it is not clear how updates of tables are different from insertions and deletions caused by the movement of the sliding windows. Second, many interesting queries need to store state, which must be kept up-to-date as time goes on. Therefore, query processing efficiency depends highly on the amount of overhead involved in state maintenance.In this paper, we show that the above issues can be solved by understanding the update patterns of continuous queries and exploiting them during query processing. We propose a classification that defines four types of update characteristics. Using our classification, we present a definition of continuous query semantics that clearly states the role of relations. We then propose the notion of update-pattern-aware query processing, where physical implementations of query operators, including the data structures used for storing intermediate state, vary depending on the update patterns of their inputs and outputs. When tested on IP traffic logs, our update-pattern-aware query plans routinely outperform the existing techniques by an order of magnitude.