HTC scientific computing in a distributed cloud environment

Authors:
Randall Sobie;Ashok Agarwal;Ian Gable;Colin Leavett-Brown;Michael Paterson;Ryan Taylor;Andre Charbonneau;Roger Impey;Wayne Podiama
Affiliations:
University of Victoria, Victoria, Canada;University of Victoria, Victoria, Canada;University of Victoria, Victoria, Canada;University of Victoria, Victoria, Canada;University of Victoria, Victoria, Canada;University of Victoria, Victoria, Canada;National Research Council, Ottawa, Canada;National Research Council, Ottawa, Canada;National Research Council, Ottawa, Canada
Venue:
Proceedings of the 4th ACM workshop on Scientific cloud computing
Year:
2013

Citing 9
Cited 0

The grid: blueprint for a new computing infrastructure

The grid: blueprint for a new computing infrastructure
Xen and the art of virtualization

SOSP '03 Proceedings of the nineteenth ACM symposium on Operating systems principles
Distributed computing in practice: the Condor experience: Research Articles

Concurrency and Computation: Practice & Experience - Grid Performance
Single sign-on for java web start applications using myproxy

Proceedings of the 3rd ACM workshop on Secure web services
Sky Computing

IEEE Internet Computing
Belle-DIRAC Setup for Using Amazon Elastic Compute Cloud

Journal of Grid Computing
Magellan: experiences from a science cloud

Proceedings of the 2nd international workshop on Scientific cloud computing
CernVM-FS: delivering scientific software to globally distributed computing resources

Proceedings of the first international workshop on Network-aware data management
FutureGrid Image Repository: A Generic Catalog and Storage System for Heterogeneous Virtual Machine Images

CLOUDCOM '11 Proceedings of the 2011 IEEE Third International Conference on Cloud Computing Technology and Science

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper describes the use of a distributed cloud computing system for high-throughput computing (HTC) scientific applications. The distributed cloud computing system is composed of a number of separate Infrastructure-as-a-Service (IaaS) clouds that are utilized in a unified infrastructure. The distributed cloud has been in production-quality operation for two years with approximately 500,000 completed jobs where a typical workload has 500 simultaneous embarrassingly-parallel jobs that run for approximately 12 hours. We review the design and implementation of the system which is based on pre-existing components and a number of custom components. We discuss the operation of the system, and describe our plans for the expansion to more sites and increased computing capacity.