The greedy load sharing algorithm
Journal of Parallel and Distributed Computing
Semi-Distributed Load Balancing for Massively Parallel Multicomputer Systems
IEEE Transactions on Software Engineering
Distributed computation on graphs: shortest path algorithms
Communications of the ACM
Creating Services with Hard Guarantees from Cycle-Harvesting Systems
CCGRID '03 Proceedings of the 3st International Symposium on Cluster Computing and the Grid
QoS-Assured Service Composition in Managed Service Overlay Networks
ICDCS '03 Proceedings of the 23rd International Conference on Distributed Computing Systems
Gigascope: a stream database for network applications
Proceedings of the 2003 ACM SIGMOD international conference on Management of data
BRITE: An Approach to Universal Topology Generation
MASCOTS '01 Proceedings of the Ninth International Symposium in Modeling, Analysis and Simulation of Computer and Telecommunication Systems
Aurora: a new model and architecture for data stream management
The VLDB Journal — The International Journal on Very Large Data Bases
sFlow: Towards Resource-Efficient and Agile Service Federation in Service Overlay Networks
ICDCS '04 Proceedings of the 24th International Conference on Distributed Computing Systems (ICDCS'04)
Optimal Resource-Aware Deployment Planning for Component-Based Distributed Applications
HPDC '04 Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing
GATES: A Grid-Based Middleware for Processing Distributed Data Streams
HPDC '04 Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing
JiST: an efficient approach to simulation using virtual machines: Research Articles
Software—Practice & Experience
Resource-Aware Distributed Stream Management Using Dynamic Overlays
ICDCS '05 Proceedings of the 25th IEEE International Conference on Distributed Computing Systems
Network-Aware Operator Placement for Stream-Processing Systems
ICDE '06 Proceedings of the 22nd International Conference on Data Engineering
The Computational and Storage Potential of Volunteer Computing
CCGRID '06 Proceedings of the Sixth IEEE International Symposium on Cluster Computing and the Grid
Journal of Parallel and Distributed Computing
HPCS '07 Proceedings of the 21st International Symposium on High Performance Computing Systems and Applications
Monitoring streams: a new class of data management applications
VLDB '02 Proceedings of the 28th international conference on Very Large Data Bases
Optimal Load Balancing in a Multiple Processor System with Many Job Classes
IEEE Transactions on Software Engineering
Challenges in dependable internet-scale stream processing
Proceedings of the 2nd workshop on Dependable distributed data management
Fast and Highly-Available Stream Processing over Wide Area Networks
ICDE '08 Proceedings of the 2008 IEEE 24th International Conference on Data Engineering
Managing opportunistic and dedicated resources in a bi-modal service deployment architecture
Managing opportunistic and dedicated resources in a bi-modal service deployment architecture
Distributed multimedia service composition with statistical QoS assurances
IEEE Transactions on Multimedia
Path selection in user-controlled circuit-switched optical networks
Optical Switching and Networking
| Hi-index | 0.00 |
This paper presents resource management techniques for allocating communication and computational resources in a distributed stream processing platform. The platform is designed to exploit the synergy of two classes of network connections-dedicated and opportunistic. Previous studies we conducted have demonstrated the benefits of such bi-modal resource organization that combines small pools of dedicated computers with a very large pool of opportunistic computing capacities of idle computers to serve high throughput computing applications. This paper extends the idea of bi-modal resource organization into the management of communication resources. Since distributed stream processing applications demand large volume of data transmission between processing sites at a consistent rate, adequate control over the network resources is important to ensure a steady flow of processing. The system model used in this paper is a platform where stream processing servers at distributed sites are interconnected with a combination of dedicated and opportunistic communication links. Two pertinent resource allocation problems are analyzed in detail and solved using decentralized algorithms. One is mapping of the processing and the communication tasks of the stream processing workload on the processing and the communication resources of the platform. The other is the dynamic re-allocation of the communication links due to variations in the capacity of the opportunistic communication links. Overall optimization goal of the allocations is higher task throughput and better utilization of the expensive dedicated links without deviating much from the timely completion of the tasks. The algorithms are evaluated through extensive simulation with a model based on realistic observations. The results demonstrate that the algorithms are able to exploit the synergy of bi-modal communication links towards achieving the optimization goals.