Congestion avoidance and control
SIGCOMM '88 Symposium proceedings on Communications architectures and protocols
The Quadtree and Related Hierarchical Data Structures
ACM Computing Surveys (CSUR)
Controlling high bandwidth aggregates in the network
ACM SIGCOMM Computer Communication Review
Design and Analysis of Parametric Query Optimization Algorithms
VLDB '98 Proceedings of the 24rd International Conference on Very Large Data Bases
VLDB '92 Proceedings of the 18th International Conference on Very Large Data Bases
Aurora: a new model and architecture for data stream management
The VLDB Journal — The International Journal on Very Large Data Bases
Load Shedding for Aggregation Queries over Data Streams
ICDE '04 Proceedings of the 20th International Conference on Data Engineering
Static optimization of conjunctive queries with sliding windows over infinite streams
SIGMOD '04 Proceedings of the 2004 ACM SIGMOD international conference on Management of data
Highly available, fault-tolerant, parallel dataflows
SIGMOD '04 Proceedings of the 2004 ACM SIGMOD international conference on Management of data
Data Triage: An Adaptive Architecture for Load Shedding in TelegraphCQ
ICDE '05 Proceedings of the 21st International Conference on Data Engineering
Network-Aware Operator Placement for Stream-Processing Systems
ICDE '06 Proceedings of the 22nd International Conference on Data Engineering
Adaptive Control of Extreme-scale Stream Processing Systems
ICDCS '06 Proceedings of the 26th IEEE International Conference on Distributed Computing Systems
Window-aware load shedding for aggregation queries over data streams
VLDB '06 Proceedings of the 32nd international conference on Very large data bases
OverQos: an overlay based architecture for enhancing internet Qos
NSDI'04 Proceedings of the 1st conference on Symposium on Networked Systems Design and Implementation - Volume 1
Salamander: a push-based distribution substrate for internet applications
USITS'97 Proceedings of the USENIX Symposium on Internet Technologies and Systems on USENIX Symposium on Internet Technologies and Systems
Parametric query optimization for linear and piecewise linear cost functions
VLDB '02 Proceedings of the 28th international conference on Very Large Data Bases
An efficient and resilient approach to filtering and disseminating streaming data
VLDB '03 Proceedings of the 29th international conference on Very large data bases - Volume 29
Load shedding in a data stream manager
VLDB '03 Proceedings of the 29th international conference on Very large data bases - Volume 29
MM '08 Proceedings of the 16th ACM international conference on Multimedia
SODA: an optimizing scheduler for large-scale stream-based distributed computer systems
Proceedings of the 9th ACM/IFIP/USENIX International Conference on Middleware
Robust network monitoring in the presence of non-cooperative traffic queries
Computer Networks: The International Journal of Computer and Telecommunications Networking
Optimized union of non-disjoint distributed data sets
Proceedings of the 12th International Conference on Extending Database Technology: Advances in Database Technology
Dynamic Query Processing for P2P Data Services in the Cloud
DEXA '09 Proceedings of the 20th International Conference on Database and Expert Systems Applications
Distributed processing of continuous join queries using DHT networks
Proceedings of the 2009 EDBT/ICDT Workshops
Resource-adaptive semantic concept detection using ensemble classifiers
ICME'09 Proceedings of the 2009 IEEE international conference on Multimedia and Expo
Resource-adaptive multimedia analysis on stream mining systems
ICME'09 Proceedings of the 2009 IEEE international conference on Multimedia and Expo
Tuning QoD in stream processing engines
ADC '10 Proceedings of the Twenty-First Australasian Conference on Database Technologies - Volume 104
Load Balancing Content-Based Publish/Subscribe Systems
ACM Transactions on Computer Systems (TOCS)
In-situ MapReduce for log processing
USENIXATC'11 Proceedings of the 2011 USENIX conference on USENIX annual technical conference
Pattern rewriting framework for event processing optimization
Proceedings of the 5th ACM international conference on Distributed event-based system
Program analysis for event-based distributed systems
Proceedings of the 5th ACM international conference on Distributed event-based system
Mining large distributed log data in near real time
SLAML '11 Managing Large-scale Systems via the Analysis of System Logs and the Application of Machine Learning Techniques
UpStream: storage-centric load management for streaming applications with update semantics
The VLDB Journal — The International Journal on Very Large Data Bases
Guarantees for decentralized event correlation
Proceedings of the 8th Middleware Doctoral Symposium
In-situ MapReduce for log processing
HotCloud'11 Proceedings of the 3rd USENIX conference on Hot topics in cloud computing
Load shedding in data stream management systems using application semantics
BNCOD'10 Proceedings of the 27th British national conference on Data Security and Security Data
FAIDECS: fair decentralized event correlation
Middleware'11 Proceedings of the 12th ACM/IFIP/USENIX international conference on Middleware
Stormy: an elastic and highly available streaming service in the cloud
Proceedings of the 2012 Joint EDBT/ICDT Workshops
A goal-oriented programming framework for grid sensor networks with reconfigurable embedded nodes
ACM Transactions on Embedded Computing Systems (TECS)
Building user-defined runtime adaptation routines for stream processing applications
Proceedings of the VLDB Endowment
An adaptive algorithm for finding frequent sets in landmark windows
SUM'12 Proceedings of the 6th international conference on Scalable Uncertainty Management
FAIDECS: fair decentralized event correlation
Proceedings of the 12th International Middleware Conference
QoS-aware optimization of sensor network queries
The VLDB Journal — The International Journal on Very Large Data Bases
| Hi-index | 0.00 |
In distributed stream processing environments, large numbers of continuous queries are distributed onto multiple servers. When one or more of these servers become overloaded due to bursty data arrival, excessive load needs to be shed in order to preserve low latency for the query results. Because of the load dependencies among the servers, load shedding decisions on these servers must be well-coordinated to achieve end-to-end control on the output quality. In this paper, we model the distributed load shedding problem as a linear optimization problem, for which we propose two alternative solution approaches: a solver-based centralized approach, and a distributed approach based on metadata aggregation and propagation, whose centralized implementation is also available. Both of our solutions are based on generating a series of load shedding plans in advance, to be used under certain input load conditions. We have implemented our techniques as part of the Borealis distributed stream processing system. We present experimental results from our prototype implementation showing the performance of these techniques under different input and query workloads.