Design, implementation, and evaluation of the linear road bnchmark on the stream processing core
Proceedings of the 2006 ACM SIGMOD international conference on Management of data
Multi-site cooperative data stream analysis
ACM SIGOPS Operating Systems Review
SPC: a distributed, scalable platform for data mining
Proceedings of the 4th international workshop on Data mining standards, services and platforms
Staying FIT: efficient load shedding techniques for distributed stream processing
VLDB '07 Proceedings of the 33rd international conference on Very large data bases
Storage optimization for large-scale distributed stream-processing systems
ACM Transactions on Storage (TOS)
EventScript: an event-processing language based on regular expressions with actions
Proceedings of the 2008 ACM SIGPLAN-SIGBED conference on Languages, compilers, and tools for embedded systems
Minimizing latency and memory in DSMS: a unified approach to quasi-optimal scheduling
SSPS '08 Proceedings of the 2nd international workshop on Scalable stream processing system
Cluster Computing
Embedding intelligent decision making within complex dynamic environments
Artificial Intelligence Review
Toward massive query optimization in large-scale distributed stream systems
Proceedings of the 9th ACM/IFIP/USENIX International Conference on Middleware
Robust network monitoring in the presence of non-cooperative traffic queries
Computer Networks: The International Journal of Computer and Telecommunications Networking
XenSocket: a high-throughput interdomain transport for virtual machines
Proceedings of the ACM/IFIP/USENIX 2007 International Conference on Middleware
CLASP: collaborating, autonomous stream processing systems
Proceedings of the ACM/IFIP/USENIX 2007 International Conference on Middleware
Proceedings of the 2nd Workshop on High Performance Computational Finance
An empirical study of high availability in stream processing systems
Proceedings of the 10th ACM/IFIP/USENIX International Conference on Middleware
Context-aware dynamic reconfiguration of mobile patient monitoring systems
ISWPC'09 Proceedings of the 4th international conference on Wireless pervasive computing
Distributed stream join query processing with semijoins
Distributed and Parallel Databases
Scheduling intense applications most 'surprising' first
Proceedings of the 2010 ACM Symposium on Applied Computing
Smart distribution of bio-signal processing tasks in m-health
OTM'07 Proceedings of the 2007 OTM confederated international conference on On the move to meaningful internet systems - Volume Part I
XenSocket: a high-throughput interdomain transport for virtual machines
MIDDLEWARE2007 Proceedings of the 8th ACM/IFIP/USENIX international conference on Middleware
CLASP: collaborating, autonomous stream processing systems
MIDDLEWARE2007 Proceedings of the 8th ACM/IFIP/USENIX international conference on Middleware
Monalytics: online monitoring and analytics for managing large scale data centers
Proceedings of the 7th international conference on Autonomic computing
Proceedings of the ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Evaluation of streaming aggregation on parallel hardware architectures
Proceedings of the Fourth ACM International Conference on Distributed Event-Based Systems
Distributed resource allocation for synchronous fork and join processing networks
INFOCOM'10 Proceedings of the 29th conference on Information communications
Complex real-time environmental monitoring of the Hudson river and estuary system
IBM Journal of Research and Development
OLIC: online information compression for scalable hosting infrastructure monitoring
Proceedings of the Nineteenth International Workshop on Quality of Service
Flow: A Stream Processing System Simulator
PADS '10 Proceedings of the 2010 IEEE Workshop on Principles of Advanced and Distributed Simulation
Utility optimal scheduling in processing networks
Performance Evaluation
M-TOP: multi-target operator placement of query graphs for data streams
Proceedings of the 15th Symposium on International Database Engineering & Applications
Processing flows of information: From data stream to complex event processing
ACM Computing Surveys (CSUR)
Software—Practice & Experience
Journal of Computer and System Sciences
Distributed Adaptive Windowed Stream Join Processing
International Journal of Distributed Systems and Technologies
Dynamic instance queuing in process-aware information systems
Proceedings of the 28th Annual ACM Symposium on Applied Computing
DANBI: dynamic scheduling of irregular stream programs for many-core systems
PACT '13 Proceedings of the 22nd international conference on Parallel architectures and compilation techniques
A catalog of stream processing optimizations
ACM Computing Surveys (CSUR)
Proceedings Demo & Poster Track of ACM/IFIP/USENIX International Middleware Conference
An event-based platform for collaborative threats detection and monitoring
Information Systems
Hi-index | 0.00 |
Distributed stream processing systems offer a highly scalable and dynamically configurable platform for time-critical applications ranging from real-time, exploratory data mining to high performance transaction processing. Resource management for distributed stream processing systems is complicated by a number of factorsprocessing elements are constrained by their producer-consumer relationships, data and processing rates can be highly bursty, and traditional measures of effectiveness, such as utilization, can be misleading. In this paper, we propose a novel distributed, adaptive control algorithm that maximizes weighted throughput while ensuring stable operation in the face of highly bursty workloads. Our algorithm is designed to meet the challenges of extreme-scale stream processing systems, where overprovisioning is not an option, by making the best use of resources even when the proffered load is greater than available resources. We have implemented our algorithm in a real-world distributed stream processing system and a simulation environment. Our results show that our algorithm is not only self-stabilizing and robust to errors, but also outperforms traditional approaches over a broad range of buffer sizes, processing graphs, and burstiness types and levels.