Development and performance study of a zero-copy file transfer mechanism for VIA-Based PC cluster systems

  • Authors:

  • Sejin Park;Sang-Hwa Chung;In-Su Yoon

  • Affiliations:

  • Telecommunication Network Business, Samsung Electronics Co.,LTD, Gumi, Korea;Department of Computer Engineering, Pusan National University, Pusan, Korea;Department of Computer Engineering, Pusan National University, Pusan, Korea

  • Venue:
  • HPCC'06 Proceedings of the Second international conference on High Performance Computing and Communications
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper presents the development and implementation of a zero-copy file transfer mechanism that improves the efficiency of file transfers for PC cluster systems using hardware-based VIA (Virtual Interface Architecture) network adapters. VIA is one of the representative user-level communication interfaces, but because there is no library for file transfer, one copy occurs between kernel buffer and user buffers. Our mechanism presents a file transfer primitive that does not require the file system to be modified and allows the NIC to transfer data from the kernel buffer to the remote node directly without copying. To do this, we have developed a hardware-based VIA network adapter, which supports the PCI 64bit/66MHz bus and Gigabit Ethernet, as a NIC, and implemented a zero-copy file transfer mechanism. The experimental results show that the overhead of data coy and context switching in the sender is greatly reduced and the CPU utilization of the sender is reduced to 30% ~ 40% of the VIA send/receive mechanism. We demonstrate the performance of the zero-copy file transfer mechanism experimentally, and compare the results with those from existing file transfer mechanisms.