StorkCloud: data transfer scheduling and optimization as a service
Proceedings of the 4th ACM workshop on Scientific cloud computing
Dynamic protocol tuning algorithms for high performance data transfers
Euro-Par'13 Proceedings of the 19th international conference on Parallel Processing
| Hi-index | 0.00 |
An important bottleneck for data-intensive scalable computing systems is efficient utilization of the network links that connect the collaborating institutions with their remote partners, data sources, and computational sites. To alleviate this bottleneck, we propose an application-layer throughput optimization model based on parallel stream number prediction. This new model extends our two previous models (Partial C-order and Full Second-order) to achieve higher accuracy and lower overhead predictions. Our new model, called Full C-order, outperforms both of our previous models as well as the most relevant model by others, the Partial Second-order, in terms of both accuracy and efficiency. We test and compare these four models on emulated testbeds and on production environments using a wide variety of data set sizes, RTT, and bandwidth combinations. Our comprehensive experiments confirm the superiority of our new model to the other three models.