Sockets Direct Protocol over InfiniBand in clusters: is it beneficial?

Authors:
P. Balaji;S. Narravula;K. Vaidyanathan;S. Krishnamoorthy;J. Wu;D. K. Panda
Affiliations:
Comput. & Inf. Sci., Ohio State Univ., Columbus, OH, USA;Comput. & Inf. Sci., Ohio State Univ., Columbus, OH, USA;Comput. & Inf. Sci., Ohio State Univ., Columbus, OH, USA;Comput. & Inf. Sci., Ohio State Univ., Columbus, OH, USA;Comput. & Inf. Sci., Ohio State Univ., Columbus, OH, USA;Comput. & Inf. Sci., Ohio State Univ., Columbus, OH, USA
Venue:
ISPASS '04 Proceedings of the 2004 IEEE International Symposium on Performance Analysis of Systems and Software
Year:
2004

Citing 0
Cited 8

Exploiting NIC architectural support for enhancing IP-based protocols on high-performance networks

Journal of Parallel and Distributed Computing - Special issue: Design and performance of networks for super-, cluster-, and grid-computing: Part II
Future networking for scalable I/O

PDCN'06 Proceedings of the 24th IASTED international conference on Parallel and distributed computing and networks
Design issues and performance comparisons in supporting the sockets interface over user-level communication architecture

The Journal of Supercomputing
Impact of protocol overheads on network throughput over high-speed interconnects: measurement, analysis, and improvement

The Journal of Supercomputing
An SSL Back-End Forwarding Scheme in Cluster-Based Web Servers

IEEE Transactions on Parallel and Distributed Systems
Sockets direct protocol for hybrid network stacks: a case study with iWARP over 10G Ethernet

HiPC'08 Proceedings of the 15th international conference on High performance computing
Asynchronous zero-copy communication for synchronous sockets in the sockets direct protocol (SDP) over InfiniBand

IPDPS'06 Proceedings of the 20th international conference on Parallel and distributed processing
Designing next generation data-centers with advanced communication protocols and systems services

IPDPS'06 Proceedings of the 20th international conference on Parallel and distributed processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

The Sockets Direct Protocol (SDP) had been proposed recently in order to enable sockets based applications to take advantage of the enhanced features provided by InfiniBand architecture. In this paper, we study the benefits and limitations of an implementation of SDP. We first analyze the performance of SDP based on a detailed suite of micro-benchmarks. Next, we evaluate it on two different real application domains: (1) A multitier data-center environment and (2) A Parallel Virtual File System (PVFS). Our micro-benchmark results show that SDP is able to provide up to 2.7 times better bandwidth as compared to the native sockets implementation over InfiniBand (IPoIB) and significantly better latency for large message sizes. Our experimental results also show that SDP is able to achieve a considerably higher performance (improvement of up to 2.4 times) as compared to IPoIB in the PVFS environment. In the data-center environment, SDP outperforms IPoIB for large file transfers inspite of currently being limited by a high connection setup time. However, this limitation is entirely implementation specific and as the InfiniBand software and hardware products are rapidly maturing, we expect this limitation to be overcome soon. Based on this, we have shown that the projected performance for SDP, without the connection setup time, can outperform IPoIB for small message transfers as well.