Proceedings of the ACM SIGCOMM '98 conference on Applications, technologies, architectures, and protocols for computer communication
Differentiated end-to-end Internet services using a weighted proportional fair sharing TCP
ACM SIGCOMM Computer Communication Review
Using MPI-2: Advanced Features of the Message Passing Interface
Using MPI-2: Advanced Features of the Message Passing Interface
The End-to-End Performance Effects of Parallel TCP Sockets on a Lossy Wide-Area Network
IPDPS '02 Proceedings of the 16th International Parallel and Distributed Processing Symposium
Scalable Socket Buffer Tuning for High-Performance Web Servers
ICNP '01 Proceedings of the Ninth International Conference on Network Protocols
Modeling and Taming Parallel TCP on the Wide Area Network
IPDPS '05 Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Papers - Volume 01
The Globus Striped GridFTP Framework and Server
SC '05 Proceedings of the 2005 ACM/IEEE conference on Supercomputing
A data transfer framework for large-scale science experiments
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
Prediction of Optimal Parallelism Level in Wide Area Data Transfers
IEEE Transactions on Parallel and Distributed Systems
FAST TCP: from theory to experiments
IEEE Network: The Magazine of Global Internetworking
StorkCloud: data transfer scheduling and optimization as a service
Proceedings of the 4th ACM workshop on Scientific cloud computing
Modeling throughput sampling size for a cloud-hosted data scheduling and optimization service
Future Generation Computer Systems
Dynamic protocol tuning algorithms for high performance data transfers
Euro-Par'13 Proceedings of the 19th international conference on Parallel Processing
| Hi-index | 0.00 |
The rapidly advancing optical networking technology allows us high-bandwidth connectivity up to 100Gbps these days. However, the end-users and their applications can only observe a fraction of this available bandwidth capacity due to inefficient transport protocols and other end-system bottlenecks such as disk I/O limitations, processor speed, and NIC restrictions. In this paper, we present a novel network-aware end-to-end throughput prediction and optimization framework which provides us with the best parameter combination (i.e. parallel stream, disk, and CPU numbers) to achieve the highest end-to-end throughput between two end-systems (i.e. clusters, data centers, parallel disk systems) possible. Our experiments show that the model and algorithm we have developed enable us to achieve close-to-optimal end-to-end throughput performance with negligible sampling and prediction overhead.