GMBlock: Optimizing data movement in a block-level storage sharing system over Myrinet

  • Authors:

  • Evangelos Koukis;Anastassios Nanos;Nectarios Koziris

  • Affiliations:

  • School of Electrical and Computer Engineering, Computing Systems Laboratory, National Technical University of Athens, Zografou, Greece 15780;School of Electrical and Computer Engineering, Computing Systems Laboratory, National Technical University of Athens, Zografou, Greece 15780;School of Electrical and Computer Engineering, Computing Systems Laboratory, National Technical University of Athens, Zografou, Greece 15780

  • Venue:
  • Cluster Computing
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

We present gmblock, a block-level storage sharing system over Myrinet which uses an optimized I/O path to transfer data directly between the storage medium and the network, bypassing the host CPU and main memory bus of the storage server. It is device driver independent and retains the protection and isolation features of the OS. We evaluate the performance of a prototype gmblock server and find that: (a) the proposed techniques eliminate memory and peripheral bus contention, increasing remote I/O bandwidth significantly, in the order of 20---200% compared to an RDMA-based approach, (b) the impact of remote I/O to local computation becomes negligible, (c) the performance characteristics of RAID storage combined with limited NIC resources reduce performance. We introduce synchronized send operations to improve the degree of disk to network I/O overlapping. We deploy the OCFS2 shared-disk filesystem over gmblock and show gains for various application benchmarks, provided I/O scheduling can eliminate the disk bottleneck due to concurrent access.