Protocols for wide-area data-intensive applications: design and performance issues
SC '12 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Design and performance evaluation of NUMA-aware RDMA-based end-to-end data transfer systems
SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
End-to-end data movement using MPI-IO over routed terabits infrastructures
NDM '13 Proceedings of the Third International Workshop on Network-Aware Data Management
| Hi-index | 0.00 |
Providing high-speed data transfer is vital to various data-intensive applications in cloud computing systems. We design a middleware layer of high-speed communication based on Remote Direct Memory Access (RDMA) that serves as the common substrate to accelerate various data transfer applications in cloud computing, such as FTP, HTTP, file copy, sync and remote file I/O. This middleware offers higher end-to-end bandwidth than the traditional TCP-based alternatives, while it hides the heterogeneity of the underlying high-speed architecture. In this paper, we describe the design of this middleware, including resource abstraction, and task synchronization and scheduling. We provide a reference implementation of the file-transfer protocol over this RDMA based middleware. Our experimental results show that it outperforms several TCP-based FTP tools, such as GridFTP, while maintaining very low CPU consumption on a variety of platforms. Furthermore, these results confirm that our middleware achieves near line-speed performance in both LAN and MAN, and scales consistently from 10Gbps Ethernet to 40Gbps Ethernet and InfiniBand environments.