Optimization of data stream processing

Authors:
Janusz R. Getta;Ehsan Vossough
Affiliations:
University of Wollongong, Wollongong, NSW, Australia;University of Western Sydney, Campbelltown, NSW, Australia
Venue:
ACM SIGMOD Record
Year:
2004

Citing 16
Cited 2

Continuous queries over append-only databases

SIGMOD '92 Proceedings of the 1992 ACM SIGMOD international conference on Management of data
Online aggregation

SIGMOD '97 Proceedings of the 1997 ACM SIGMOD international conference on Management of data
Eddies: continuously adaptive query processing

SIGMOD '00 Proceedings of the 2000 ACM SIGMOD international conference on Management of data
Transactional information systems: theory, algorithms, and the practice of concurrency control and recovery

Transactional information systems: theory, algorithms, and the practice of concurrency control and recovery
Dataflow query execution in a parallel main-memory environment

PDIS '91 Proceedings of the first international conference on Parallel and distributed information systems
Models and issues in data stream systems

Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Rate-based query optimization for streaming information sources

Proceedings of the 2002 ACM SIGMOD international conference on Management of data
Continuously adaptive continuous queries over streams

Proceedings of the 2002 ACM SIGMOD international conference on Management of data
Online Dynamic Reordering for Interactive Data Processing

VLDB '99 Proceedings of the 25th International Conference on Very Large Data Bases
Chain: operator scheduling for memory minimization in data stream systems

Proceedings of the 2003 ACM SIGMOD international conference on Management of data
STREAM: the stanford stream data manager (demonstration description)

Proceedings of the 2003 ACM SIGMOD international conference on Management of data
Aurora: a data stream management system

Proceedings of the 2003 ACM SIGMOD international conference on Management of data
TelegraphCQ: continuous dataflow processing

Proceedings of the 2003 ACM SIGMOD international conference on Management of data
Fjording the Stream: An Architecture for Queries Over Streaming Sensor Data

ICDE '02 Proceedings of the 18th International Conference on Data Engineering
Monitoring streams: a new class of data management applications

VLDB '02 Proceedings of the 28th international conference on Very Large Data Bases
Tuple routing strategies for distributed eddies

VLDB '03 Proceedings of the 29th international conference on Very large data bases - Volume 29

Context-model generation for safe autonomous transport vehicles

Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems
Modeling and optimizing large-scale data flows

Future Generation Computer Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

Efficient processing of unlimited and continuously expanding sequences of data items is one of the key factors in the implementations of Data Stream Management Systems (DSMS). Analysis of stream processing at the dataflow level reveals execution plans which are not visible at a logical level. This work introduces a new model of data stream processing and discusses a number of optimization techniques applicable to this model and its implementation. The optimization techniques include applications of containers with intermediate results, analysis of data processing rates, and efficient synchronization of elementary operations on data streams. The paper also describes the translation of logical level expressions on data streams into the sets of dataflow level expressions, syntax based optimization of dataflow expression, and scheduling of concurrent computations of the dataflow expressions.