Efficient wide area data transfer protocols for 100 Gbps networks and beyond

Authors:
Ezra Kissel;Martin Swany;Brian Tierney;Eric Pouyoul
Affiliations:
Indiana University, Bloomington, IN;Indiana University, Bloomington, IN;Lawrence Berkeley National Laboratory, Berkeley, CA;Lawrence Berkeley National Laboratory, Berkeley, CA
Venue:
NDM '13 Proceedings of the Third International Workshop on Network-Aware Data Management
Year:
2013

Citing 13
Cited 0

High performance TCP in ANSNET

ACM SIGCOMM Computer Communication Review
UDT: UDP-based data transfer for high-speed wide area networks

Computer Networks: The International Journal of Computer and Telecommunications Networking
Wide-area performance profiling of 10GigE and InfiniBand technologies

Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Performance of HPC Middleware over InfiniBand WAN

ICPP '08 Proceedings of the 2008 37th International Conference on Parallel Processing
A comprehensive TCP fairness analysis in high speed networks

Computer Communications
Efficient Zero-Copy Noncontiguous I/O for Globus on InfiniBand

ICPPW '10 Proceedings of the 2010 39th International Conference on Parallel Processing Workshops
On TCP performance in a heterogeneous network: a survey

IEEE Communications Magazine
A Performance Study to Guide RDMA Programming Decisions

HPCC '12 Proceedings of the 2012 IEEE 14th International Conference on High Performance Computing and Communication & 2012 IEEE 9th International Conference on Embedded Software and Systems
Evaluating High Performance Data Transfer with RDMA-based Protocols in Wide-Area Networks

HPCC '12 Proceedings of the 2012 IEEE 14th International Conference on High Performance Computing and Communication & 2012 IEEE 9th International Conference on Embedded Software and Systems
Protocols for wide-area data-intensive applications: design and performance issues

SC '12 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Efficient data transfer protocols for big data

E-SCIENCE '12 Proceedings of the 2012 IEEE 8th International Conference on E-Science (e-Science)
Efficient LHC Data Distribution across 100Gbps Networks

SCC '12 Proceedings of the 2012 SC Companion: High Performance Computing, Networking Storage and Analysis
The Science DMZ: a network design pattern for data-intensive science

SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis

Quantified Score

Hi-index	0.00

Visualization

Abstract

Due to a number of recent technology developments, now is the right time to re-examine the use of TCP for very large data transfers. These developments include the deployment of 100 Gigabit per second (Gbps) network backbones, hosts that can easily manage 40 Gbps, and higher, data transfers, the Science DMZ model, the availability of virtual circuit technology, and wide-area Remote Direct Memory Access (RDMA) protocols. In this paper we show that RDMA works well over wide-area virtual circuits, and uses much less CPU than TCP or UDP. We also characterize the limitations of RDMA in the presence of other traffic, including competing RDMA flows. We conclude that RDMA for Science DMZ to Science DMZ transfers of massive data is a viable and desirable option for high-performance data transfer.