Eddies: continuously adaptive query processing
SIGMOD '00 Proceedings of the 2000 ACM SIGMOD international conference on Management of data
Online Dynamic Reordering for Interactive Data Processing
VLDB '99 Proceedings of the 25th International Conference on Very Large Data Bases
Dynamic Pipeline Scheduling for Improving Interactive Query Performance
Proceedings of the 27th International Conference on Very Large Data Bases
Exploiting Punctuation Semantics in Continuous Data Streams
IEEE Transactions on Knowledge and Data Engineering
Gigascope: a stream database for network applications
Proceedings of the 2003 ACM SIGMOD international conference on Management of data
Aurora: a new model and architecture for data stream management
The VLDB Journal — The International Journal on Very Large Data Bases
Holistic UDAFs at streaming speeds
SIGMOD '04 Proceedings of the 2004 ACM SIGMOD international conference on Management of data
Evaluating window joins over punctuated streams
Proceedings of the thirteenth ACM international conference on Information and knowledge management
Flexible time management in data stream systems
PODS '04 Proceedings of the twenty-third ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
A robust system for accurate real-time summaries of internet traffic
SIGMETRICS '05 Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Fault-tolerance in the Borealis distributed stream processing system
Proceedings of the 2005 ACM SIGMOD international conference on Management of data
Semantics and evaluation techniques for window aggregates in data streams
Proceedings of the 2005 ACM SIGMOD international conference on Management of data
A heartbeat mechanism and its application in gigascope
VLDB '05 Proceedings of the 31st international conference on Very large data bases
Optimizing in-order execution of continuous queries over streamed sensor data
SSDBM'2005 Proceedings of the 17th international conference on Scientific and statistical database management
Punctuated data streams
On-the-fly sharing for streamed aggregation
Proceedings of the 2006 ACM SIGMOD international conference on Management of data
The CQL continuous query language: semantic foundations and query execution
The VLDB Journal — The International Journal on Very Large Data Bases
Safety guarantee of continuous join queries over punctuated data streams
VLDB '06 Proceedings of the 32nd international conference on Very large data bases
Maximizing the output rate of multi-way join queries over streaming information sources
VLDB '03 Proceedings of the 29th international conference on Very large data bases - Volume 29
Scheduling for shared window joins over data streams
VLDB '03 Proceedings of the 29th international conference on Very large data bases - Volume 29
Processing sliding window multi-joins in continuous queries over data streams
VLDB '03 Proceedings of the 29th international conference on Very large data bases - Volume 29
Memory-limited execution of windowed stream joins
VLDB '04 Proceedings of the Thirtieth international conference on Very large data bases - Volume 30
Resource sharing in continuous sliding-window aggregates
VLDB '04 Proceedings of the Thirtieth international conference on Very large data bases - Volume 30
CAPE: continuous query engine with heterogeneous-grained adaptivity
VLDB '04 Proceedings of the Thirtieth international conference on Very large data bases - Volume 30
Fast and Highly-Available Stream Processing over Wide Area Networks
ICDE '08 Proceedings of the 2008 IEEE 24th International Conference on Data Engineering
Tagging stream data for rich real-time services
Proceedings of the VLDB Endowment
On-the-fly progress detection in iterative stream queries
Proceedings of the VLDB Endowment
Toward Formal Semantics for Data and Schema Evolution in Data Stream Management Systems
ER '09 Proceedings of the ER 2009 Workshops (CoMoL, ETheCoM, FP-UML, MOST-ONISW, QoIS, RIGiM, SeCoGIS) on Advances in Conceptual Modeling - Challenging Perspectives
Stream schema: providing and exploiting static metadata for data stream processing
Proceedings of the 13th International Conference on Extending Database Technology
Continuous analytics over discontinuous streams
Proceedings of the 2010 ACM SIGMOD International Conference on Management of data
An extensible test framework for the Microsoft StreamInsight query processor
Proceedings of the Third International Workshop on Testing Database Systems
Framing the question: detecting and filling spatial-temporal windows
Proceedings of the ACM SIGSPATIAL International Workshop on GeoStreaming
Real-time route planning with stream processing systems: a case study for the city of Lucerne
Proceedings of the 2nd ACM SIGSPATIAL International Workshop on GeoStreaming
Capturing episodes: may the frame be with you
Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems
Measuring performance of complex event processing systems
TPCTC'11 Proceedings of the Third TPC Technology conference on Topics in Performance Evaluation, Measurement and Characterization
Predictive analytics with surveillance big data
Proceedings of the 1st ACM SIGSPATIAL International Workshop on Analytics for Big Geospatial Data
Proceedings of the 7th ACM international conference on Distributed event-based systems
A new operator for efficient stream-relation join processing in data streaming engines
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles
ACM SIGOPS 24th Symposium on Operating Systems Principles
Naiad: a timely dataflow system
Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles
Using a real-time top-k algorithm to mine the most frequent items over multiple streams
ICIC'13 Proceedings of the 9th international conference on Intelligent Computing Theories
MillWheel: fault-tolerant stream processing at internet scale
Proceedings of the VLDB Endowment
Hi-index | 0.00 |
Many stream-processing systems enforce an order on data streams during query evaluation to help unblock blocking operators and purge state from stateful operators. Such in-order processing (IOP) systems not only must enforce order on input streams, but also require that query operators preserve order. This order-preserving requirement constrains the implementation of stream systems and incurs significant performance penalties, particularly for memory consumption. Especially for high-performance, potentially distributed stream systems, the cost of enforcing order can be prohibitive. We introduce a new architecture for stream systems, out-of-order processing (OOP), that avoids ordering constraints. The OOP architecture frees stream systems from the burden of order maintenance by using explicit stream progress indicators, such as punctuation or heartbeats, to unblock and purge operators. We describe the implementation of OOP stream systems and discuss the benefits of this architecture in depth. For example, the OOP approach has proven useful for smoothing workload bursts caused by expensive end-of-window operations, which can overwhelm internal communication paths in IOP approaches. We have implemented OOP in two stream systems, Gigascope and NiagaraST. Our experimental study shows that the OOP approach can significantly outperform IOP in a number of aspects, including memory, throughput and latency.