On Composing Stream Applications in Peer-to-Peer Environments
IEEE Transactions on Parallel and Distributed Systems
ViCo: an adaptive distributed video correlation system
MULTIMEDIA '06 Proceedings of the 14th annual ACM international conference on Multimedia
Biologically-inspired distributed middleware management for stream processing systems
Proceedings of the 9th ACM/IFIP/USENIX International Conference on Middleware
Adaptive load diffusion for stream joins
Proceedings of the ACM/IFIP/USENIX 2005 International Conference on Middleware
Synergy: sharing-aware component composition for distributed stream processing systems
Proceedings of the ACM/IFIP/USENIX 2006 International Conference on Middleware
Towards secure dataflow processing in open distributed systems
Proceedings of the 2009 ACM workshop on Scalable trusted computing
Placement of replicated tasks for distributed stream processing systems
Proceedings of the Fourth ACM International Conference on Distributed Event-Based Systems
Synergy: sharing-aware component composition for distributed stream processing systems
Middleware'06 Proceedings of the 7th ACM/IFIP/USENIX international conference on Middleware
Operator placement with QoS constraints for distributed stream processing
Proceedings of the 7th International Conference on Network and Services Management
Adaptive load diffusion for stream joins
Middleware'05 Proceedings of the ACM/IFIP/USENIX 6th international conference on Middleware
| Hi-index | 0.00 |
Stream processing has become increasingly important with emergence of stream applications such as audio/video surveillance, stock price tracing, and sensor data analysis. A challenging problem is to provide optimal component composition in a distributed stream processing environment. The goal of optimal component composition is to achieve load balancing subject to multiple function, resource, and quality-of-service (QoS) constraints while composing stream applications. In this paper, we present an adaptive composition probing (ACP) approach to the problem. Different from previous work, ACP provides a new hybrid approach that combines distributed composition probing with coarse-grain global state management. Guided by the coarse-grain global state information, ACP selectively probes a subset of candidate components to discover an approximately optimal component composition. Further, ACP is self-tuning, which can adaptively adjust the number of probes to maintain a specified composition performance target (i.e., composition success rate) in a dynamic stream environment. While the optimal component composition problem is NP-hard, our ACP approach provides an adaptive polynomial approximation solution. We have conducted extensive simulation experiments to show the efficiency, scalability, and adaptability of the ACP approach by comparing with other alternative solutions.