Firmware-Level Latency Analysis on a Gigabit Network

Authors:
Hyun-wook Jin;Chuck Yoo;Jin-young Choi
Affiliations:
Department of Computer Science and Engineering, Korea University, 1, 5Ka, Anam-Dong, Sungbuk-Ku, Seoul, 136-701 Korea hwjin@os.korea.ac.kr;Department of Computer Science and Engineering, Korea University, 1,5Ka, Anam-Dong, Sungbuk-Ku, Seoul, 136-701 Korea hxy@os.korea.ac.kr;Department of Computer Science and Engineering, Korea University, 1, 5Ka, Anam-Dong, Sungbuk-Ku, Seoul, 136-701 Korea choi@formal.korea.ac.kr
Venue:
The Journal of Supercomputing
Year:
2003

Citing 13
Cited 1

Fbufs: a high-bandwidth cross-domain transfer facility

SOSP '93 Proceedings of the fourteenth ACM symposium on Operating systems principles
U-Net: a user-level network interface for parallel and distributed computing

SOSP '95 Proceedings of the fifteenth ACM symposium on Operating systems principles
High performance messaging on workstations: Illinois fast messages (FM) for Myrinet

Supercomputing '95 Proceedings of the 1995 ACM/IEEE conference on Supercomputing
PM2: a high performance communication middleware for heterogeneous network environments

Proceedings of the 2000 ACM/IEEE conference on Supercomputing
EMP: zero-copy OS-bypass NIC-driven gigabit ethernet message passing

Proceedings of the 2001 ACM/IEEE conference on Supercomputing
Myrinet: A Gigabit-per-Second Local Area Network

IEEE Micro
The Virtual Interface Architecture

IEEE Micro
Software Support for Virtual Memory-Mapped Communication

IPPS '96 Proceedings of the 10th International Parallel Processing Symposium
Bottleneck Analysis of a Gigabit Network Interface Card: Formal Verification Approach

Proceedings of the 9th International SPIN Workshop on Model Checking of Software
Stepwise Optimizations of UDP/IP on a Gigabit Network (Research Note)

Euro-Par '02 Proceedings of the 8th International Euro-Par Conference on Parallel Processing
Copy Emulation in Checksummed, Multiple-Packet Communication

INFOCOM '97 Proceedings of the INFOCOM '97. Sixteenth Annual Joint Conference of the IEEE Computer and Communications Societies. Driving the Information Revolution
Zero-copy TCP in Solaris

ATEC '96 Proceedings of the 1996 annual conference on USENIX Annual Technical Conference
Trapeze/IP: TCP/IP at near-gigabit speeds

ATEC '99 Proceedings of the annual conference on USENIX Annual Technical Conference

Impact of protocol overheads on network throughput over high-speed interconnects: measurement, analysis, and improvement

The Journal of Supercomputing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Gigabit networks are equipped with “increasingly” intelligent network interface cards, and the firmware running in the cards does various tasks related to end-to-end communication. For an accurate performance evaluation of gigabit networks, it is very important to characterize and quantify the firmware. However, the firmware has been neglected in the latency analyzes of network protocols.This paper presents an in-depth latency analysis of Myrinet. Our findings include that the major bottleneck is the network interface card itself. This is true especially for so-called lightweight user-level protocols (such as BPI of Myrinet) designed for high-speed communication. Although BPI is very lean and efficient in the host, its sending throughput becomes similar to UDP. This result is very unexpected and surprising. Through firmware-level measurements, we identify that the cause of bottleneck is the DMA performance.