Load shedding in stream databases: a control-based approach

  • Authors:

  • Yi-Cheng Tu;Song Liu;Sunil Prabhakar;Bin Yao

  • Affiliations:

  • Department of Computer Sciences, Purdue University, West Lafayette, Indiana;School of Mechanical Engineering, Purdue University, West Lafayette, Indiana;Department of Computer Sciences, Purdue University, West Lafayette, Indiana;School of Mechanical Engineering, Purdue University, West Lafayette, Indiana

  • Venue:
  • VLDB '06 Proceedings of the 32nd international conference on Very large data bases
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

In Data Stream Management Systems (DSMSs), query processing has to meet various Quality-of-Service (QoS) requirements. In many data stream applications, processing delay is the most critical quality requirement since the value of query results decreases dramatically over time. The ability to remain within a desired level of delay is significantly hampered under situations of overloading, which are common in data stream systems. When overloaded, DSMSs employ load shedding in order to meet quality requirements and keep pace with the high rate of data arrivals. Data stream applications are extremely dynamic due to bursty data arrivals and time-varying data processing costs. Current approaches ignore system status information in decision-making and consequently are unable to achieve desired control of quality under dynamic load. In this paper, we present a quality management framework that leverages well studied feedback control techniques. We discuss the design and implementation of such a framework in a real DSMS - the Borealis stream manager. Our data management framework is built on the advantages of system identification and rigorous controller analysis. Experimental results show that our solution achieves significantly fewer QoS (delay) violations with the same or lower level of data loss, as compared to current strategies utilized in DSMSs. It is also robust and bears negligible computational overhead.