Magellan: experiences from a science cloud

Authors:
Lavanya Ramakrishnan;Piotr T. Zbiegel;Scott Campbell;Rick Bradshaw;Richard Shane Canon;Susan Coghlan;Iwona Sakrejda;Narayan Desai;Tina Declerck;Anping Liu
Affiliations:
Lawrence Berkeley National Lab, Berkeley, CA, USA;Argonne National Lab, Argonne, IL, USA;Lawrence Berkeley National Lab, Berkeley, CA, USA;Argonne National Lab, Argonne, IL, USA;Lawrence Berkeley National Lab, Berkeley, CA, USA;Argonne National Lab, Argonne, IL, USA;Lawrence Berkeley National Lab, Berkeley, CA, USA;Argonne National Lab, Argonne, IL, USA;Lawrence Berkeley National Lab, Berkeley, CA, USA;Argonne National Lab, Argonne, IL, USA
Venue:
Proceedings of the 2nd international workshop on Scientific cloud computing
Year:
2011

Citing 13
Cited 12

MapReduce: simplified data processing on large clusters

OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Amazon S3 for science grids: a viable solution?

DADC '08 Proceedings of the 2008 international workshop on Data-aware distributed computing
The cost of doing science on the cloud: the Montage example

Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Scientific computing using virtual high-performance computing: a case study using the Amazon elastic computing cloud

Proceedings of the 2008 annual research conference of the South African Institute of Computer Scientists and Information Technologists on IT research in developing countries: riding the wave of technology
Can cloud computing reach the top500?

Proceedings of the combined workshops on UnConventional high performance computing workshop plus memory access workshop
CloudBurst

Bioinformatics
Cloud Computing for Science

SSDBM 2009 Proceedings of the 21st International Conference on Scientific and Statistical Database Management
VGrADS: enabling e-Science workflows on grids and clouds with fault tolerance

Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis
Scientific workflows and clouds

Crossroads - Plugging Into the Cloud
The impact of virtualization on network performance of amazon EC2 data center

INFOCOM'10 Proceedings of the 29th conference on Information communications
Finding Tropical Cyclones on a Cloud Computing Cluster: Using Parallel Virtualization for Large-Scale Climate Simulation Analysis

CLOUDCOM '10 Proceedings of the 2010 IEEE Second International Conference on Cloud Computing Technology and Science
Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud

CLOUDCOM '10 Proceedings of the 2010 IEEE Second International Conference on Cloud Computing Technology and Science
Scientific Computing in the Cloud

Computing in Science and Engineering

BlobCR: efficient checkpoint-restart for HPC applications on IaaS clouds using virtual disk image snapshots

Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis
MARIANE: MApReduce Implementation Adapted for HPC Environments

GRID '11 Proceedings of the 2011 IEEE/ACM 12th International Conference on Grid Computing
Understanding scheduling implications for scientific applications in clouds

Proceedings of the 9th International Workshop on Middleware for Grids, Clouds and e-Science
Riding the elephant: managing ensembles with hadoop

Proceedings of the 2011 ACM international workshop on Many task computing on grids and supercomputers
Parametric Optimization of Linear and Non-Linear Models via Parallel Computing to Enhance Web-Spatial DSS Interactivity

International Journal of Decision Support System Technology
HTC scientific computing in a distributed cloud environment

Proceedings of the 4th ACM workshop on Scientific cloud computing
BlobCR: Virtual disk based checkpoint-restart for HPC applications on IaaS clouds

Journal of Parallel and Distributed Computing
Failure analysis of distributed scientific workflows executing in the cloud

Proceedings of the 8th International Conference on Network and Service Management
DDS: A deadlock detection-based scheduling algorithm for workflow computations in HPC systems with storage constraints

Parallel Computing
Scalable multimedia content analysis on parallel platforms using python

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP)
Poncho: enabling smart administration of full private clouds

LISA'13 Proceedings of the 27th international conference on Large Installation System Administration
A Walking Dwarf on the Clouds

UCC '13 Proceedings of the 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Cloud resources promise to be an avenue to address new categories of scientific applications including data-intensive science applications, on-demand/surge computing, and applications that require customized software environments. However, there is a limited understanding on how to operate and use clouds for scientific applications. Magellan, a project funded through the Department of Energy's (DOE) Advanced Scientific Computing Research (ASCR) program, is investigating the use of cloud computing for science at the Argonne Leadership Computing Facility (ALCF) and the National Energy Research Scientific Computing Facility (NERSC). In this paper, we detail the experiences to date at both sites and identify the gaps and open challenges from both a resource provider as well as application perspective.