Modeling the execution semantics of stream processing engines with SECRET

Authors:
Nihal Dindar;Nesime Tatbul;Renée J. Miller;Laura M. Haas;Irina Botan
Affiliations:
ETH Zurich, Zurich, Switzerland;ETH Zurich, Zurich, Switzerland;University of Toronto, Toronto, Canada;IBM Almaden Research Center, San Jose, USA;ETH Zurich, Zurich, Switzerland
Venue:
The VLDB Journal — The International Journal on Very Large Data Bases
Year:
2013

Citing 12
Cited 0

Aurora: a new model and architecture for data stream management

The VLDB Journal — The International Journal on Very Large Data Bases
Flexible time management in data stream systems

PODS '04 Proceedings of the twenty-third ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Semantics and evaluation techniques for window aggregates in data streams

Proceedings of the 2005 ACM SIGMOD international conference on Management of data
The CQL continuous query language: semantic foundations and query execution

The VLDB Journal — The International Journal on Very Large Data Bases
Extending XQuery with window functions

VLDB '07 Proceedings of the 33rd international conference on Very large data bases
SPADE: the system s declarative stream processing engine

Proceedings of the 2008 ACM SIGMOD international conference on Management of data
Towards a streaming SQL standard

Proceedings of the VLDB Endowment
Semantics and implementation of continuous sliding window queries over data streams

ACM Transactions on Database Systems (TODS)
DejaVu: declarative pattern matching over live and archived streams of events

Proceedings of the 2009 ACM SIGMOD International Conference on Management of data
SECRET: a model for analysis of the execution semantics of stream processing systems

Proceedings of the VLDB Endowment
Semantics of data streams and operators

ICDT'05 Proceedings of the 10th international conference on Database Theory
Window specification over data streams

EDBT'06 Proceedings of the 2006 international conference on Current Trends in Database Technology

Quantified Score

Hi-index	0.00

Visualization

Abstract

There are many academic and commercial stream processing engines (SPEs) today, each of them with its own execution semantics. This variation may lead to seemingly inexplicable differences in query results. In this paper, we present SECRET, a model of the behavior of SPEs. SECRET is a descriptive model that allows users to analyze the behavior of systems and understand the results of window-based queries (with time- and tuple-based windows) for a broad range of heterogeneous SPEs. The model is the result of extensive analysis and experimentation with several commercial and academic engines. In the paper, we describe the types of heterogeneity found in existing engines and show with experiments on real systems that our model can explain the key differences in windowing behavior.