Dealing with Overload in Distributed Stream Processing Systems

Authors:
Nesime Tatbul;Stan Zdonik
Affiliations:
Brown Unversity;Brown University
Venue:
ICDEW '06 Proceedings of the 22nd International Conference on Data Engineering Workshops
Year:
2006

Citing 0
Cited 7

Window-aware load shedding for aggregation queries over data streams

VLDB '06 Proceedings of the 32nd international conference on Very large data bases
Load Management and High Availability in the Borealis Distributed Stream Processing Engine

GeoSensor Networks
NexusDS: a flexible and extensible middleware for distributed stream processing

IDEAS '09 Proceedings of the 2009 International Database Engineering & Applications Symposium
A rules-based approach for configuring chains of classifiers in real-time stream mining systems

EURASIP Journal on Advances in Signal Processing
Tool support for the design and management of context models

Information Systems
A distributed approach for optimizing cascaded classifier topologies in real-time stream mining systems

IEEE Transactions on Image Processing
Processing flows of information: From data stream to complex event processing

ACM Computing Surveys (CSUR)

Quantified Score

Hi-index	0.00

Visualization

Abstract

Overload management has been an important problem for large-scale dynamic systems. In this paper, we study this problem in the context of our Borealis distributed stream processing system. We show that server nodes must coordinate in their load shedding decisions to achieve global control on output quality. We describe a distributed load shedding approach which provides this coordination by upstream metadata aggregation and propagation. Metadata enables an upstream node to make fast local load shedding decisions which will influence its descendant nodes in the best possible way.