Xen-Based HPC: A Parallel I/O Perspective

Authors:
Weikuan Yu;Jeffrey S. Vetter
Affiliations:
-;-
Venue:
CCGRID '08 Proceedings of the 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid
Year:
2008

Citing 0
Cited 9

GViM: GPU-accelerated virtual machines

Proceedings of the 3rd ACM Workshop on System-level Virtualization for High Performance Computing
Scientific workflows and clouds

Crossroads - Plugging Into the Cloud
The impact of virtualisation on the performance and operational costs of Massively Multiplayer Online Games

International Journal of Advanced Media and Communication
Data Sharing Options for Scientific Workflows on Amazon EC2

Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis
Experiences using cloud computing for a scientific workflow application

Proceedings of the 2nd international workshop on Scientific cloud computing
Cloud versus in-house cluster: evaluating Amazon cluster compute instances for running MPI applications

State of the Practice Reports
One optimized I/O configuration per HPC application: leveraging the configurability of cloud

Proceedings of the Second Asia-Pacific Workshop on Systems
An Evaluation of the Cost and Performance of Scientific Workflows on Amazon EC2

Journal of Grid Computing
Analysis of I/O Performance on an Amazon EC2 Cluster Compute and High I/O Platform

Journal of Grid Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Virtualization using Xen-based virtual machine environment has yet to permeate the field of high performance computing (HPC). One major requirement for HPC is the availability of scalable and high performance I/O. Conventional wisdom suggests that virtualization of system services must lead to degraded performance. In this presentation, we take on a parallel I/O perspective to study the viability of Xen-based HPC for data-intensive programs. We have analyzed the overheads and migration costs for parallel I/O programs in a Xen-based virtual machine cluster. Our analysis covers PVFS-based parallel I/O over two different networking protocols: TCP-based Gigabit Ethernet and VMM-bypass InfiniBand. Our experimental results suggest that network processing in Xen-based virtualization can significantly impact the performance of Parallel I/O. By carefully tuning the networking layers, we have demonstrated the following for Xen-based HPC I/O: (1) TCP offloading can help achieve low overhead parallel I/O; (2) parallel reads and writes require different network tuning to achieve good I/O bandwidth; and (3) Xen-based HPC environment can support high performance parallel I/O with both negligible overhead and little migration cost.