Virtual Organization Clusters

• Authors: Michael A. Murphy;Michael Fenn;Sebastien Goasguen
• Venue: PDP '09 Proceedings of the 2009 17th Euromicro International Conference on Parallel, Distributed and Network-based Processing
• Year: 2009

Sharing traditional clusters based on multiprogramming systems among different Virtual Organizations (VOs) can lead to complex situations resulting from the differing software requirements of each VO. This complexity could be eliminated if each cluster computing system supported only a single VO, thereby permitting the VO to customize the operating system and software selection available on its private cluster. While dedicating entire physical clusters on the Grid to single VOs is not practical in terms of cost and scale, an equivalent separation of VOs may be accomplished by deploying clusters of Virtual Machines (VMs) in a manner that gives each VO its own virtual cluster. Such Virtual Organization Clusters (VOCs) can have numerous benefits, including isolation of VOs from one another, independence of each VOC from the underlying hardware, allocation of physical resources on a per-VO basis, and clear separation of administrative responsibilities between the physical fabric provider and the VO itself.Initial results of implementing a complete system utilizing the proposed Virtual Organization Cluster Model confirm the administrative simplicity of isolating VO software from the physical system. End-user computational jobs submitted through the Grid are executed only on the virtual cluster supporting the respective VO, and each VO has substantial administrative flexibility in terms of software choice and system configuration. Performance tests using the Kernel-based Virtual Machine (KVM) hypervisor indicated a virtualization overhead of under 10% for latency-tolerant scientific applications, such as those that would be submitted to a standard or vanilla Condor universe. Latency-sensitive applications, such as MPI, experience substantial performance degradation with virtualization overheads on the order of 60%. These results suggest that VOCs are suitable for High-Throughput Computing (HTC) applications, where real-time network performance is not critical. VOCs might also be useful for High-Performance Computing (HPC) applications if virtual network performance can be sufficiently improved.