Accommodating bursts in distributed stream processing systems

Authors:
Yannis Drougas;Vana Kalogeraki
Affiliations:
Department of Computer Science and Engineering, University of California-Riverside, USA;Department of Computer Science and Engineering, University of California-Riverside, USA
Venue:
IPDPS '09 Proceedings of the 2009 IEEE International Symposium on Parallel&Distributed Processing
Year:
2009

Citing 0
Cited 2

Flow: A Stream Processing System Simulator

PADS '10 Proceedings of the 2010 IEEE Workshop on Principles of Advanced and Distributed Simulation
Towards flexible exascale stream processing system simulation

Simulation

Quantified Score

Hi-index	0.00

Visualization

Abstract

Stream processing systems have become important, as applications like media broadcasting, sensor network monitoring and on-line data analysis increasingly rely on real-time stream processing. Such systems are often challenged by the bursty nature of the applications. In this paper, we present BARRE (Burst Accommodation through Rate REconfiguration), a system to address the problem of bursty data streams in distributed stream processing systems. Upon the emergence of a burst, BARRE dynamically reserves resources dispersed across the nodes of a distributed stream processing system, based on the requirements of each application as well as the resources available on the nodes. Our experimental results over our Synergy distributed stream processing system demonstrate the efficiency of our approach.