Network flow based resource brokering and optimization techniques for distributed data streaming over optical networks

Authors:
Cornelius Toole, Jr.;Andrei Hutanu
Affiliations:
Louisiana State University, Baton Rouge, LA;Louisiana State University, Baton Rouge, LA
Venue:
Proceedings of the 15th ACM Mardi Gras conference: From lightweight mash-ups to lambda grids: Understanding the spectrum of distributed computing requirements, applications, tools, infrastructures, interoperability, and the incremental adoption of key capabilities
Year:
2008

Citing 13
Cited 0

Network flows: theory, algorithms, and applications

Network flows: theory, algorithms, and applications
Efficient dynamic network flow algorithms

Efficient dynamic network flow algorithms
The network weather service: a distributed resource performance forecasting service for metacomputing

Future Generation Computer Systems - Special issue on metacomputing
An end-to-end approach to globally scalable network storage

Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications
Using Disk Throughput Data in Predictions of End-to-End Grid Data Transfers

GRID '02 Proceedings of the Third International Workshop on Grid Computing
Prophesy: an infrastructure for performance analysis and modeling of parallel and grid applications

ACM SIGMETRICS Performance Evaluation Review
Reliable Blast UDP: Predictable High Performance Bulk Data Transfer

CLUSTER '02 Proceedings of the IEEE International Conference on Cluster Computing
Predicting the Performance of Wide Area Data Transfers

IPDPS '02 Proceedings of the 16th International Symposium on Parallel and Distributed Processing
Characterizing and Predicting TCP Throughput on the Wide Area Network

ICDCS '05 Proceedings of the 25th IEEE International Conference on Distributed Computing Systems
Adaptive multi-resource prediction in distributed resource sharing environment

CCGRID '04 Proceedings of the 2004 IEEE International Symposium on Cluster Computing and the Grid
Distributed and collaborative visualization of large data sets using high-speed networks

Future Generation Computer Systems - IGrid 2005: The global lambda integrated facility
G-lambda and EnLIGHTened: wrapped in middleware co-allocating compute and network resources across Japan and the US

Proceedings of the first international conference on Networks for grid applications
HARC: the highly-available resource co-allocator

OTM'07 Proceedings of the 2007 OTM confederated international conference on On the move to meaningful internet systems: CoopIS, DOA, ODBASE, GADA, and IS - Volume Part II

Quantified Score

Hi-index	0.00

Visualization

Abstract

This article analyzes the problem of optimizing access to and transport of large remote data through intelligent resource selection and configuration. With the availability of high speed optical networks, the main problem for remote data access is shifting from having enough network bandwidth to having enough data ready to saturate the network when requested by the application. Network bandwidth is now higher than disk bandwidth and this gives us the possibility of utilizing multiple distributed resources to saturate the network links. We are considering two types of scenarios, one where we use only disks as data sources and a more advanced scenario where compute resources in the network can be utilized as caches to increase instantaneous throughput. The problem we are facing is choosing and configuring the resources for this scenario. This is a non-trivial problem however as we are using application-driven network resource allocation (which gives us predictability and determinism in terms of network performance) the problem becomes tractable. We discuss optimization algorithms that are applicable to this problem, and present an algorithm that divides the problem in two sub-problems that can be solved using existing network flow algorithms.