Variable-sized map and locality-aware reduce on public-resource grids

Authors:
Yen-Liang Su;Po-Cheng Chen;Jyh-Biau Chang;Ce-Kuen Shieh
Affiliations:
Institute of Computer and Communication Engineering, Department of Electrical Engineering, National Cheng Kung University, Taiwan;Institute of Computer and Communication Engineering, Department of Electrical Engineering, National Cheng Kung University, Taiwan;Department of Digital Applications, Leader University, Taiwan;Institute of Computer and Communication Engineering, Department of Electrical Engineering, National Cheng Kung University, Taiwan
Venue:
Future Generation Computer Systems
Year:
2011

Citing 13
Cited 1

Teamster-G: a grid-enabled software DSM system

CCGRID '05 Proceedings of the Fifth IEEE International Symposium on Cluster Computing and the Grid (CCGrid'05) - Volume 2 - Volume 02
Evaluating MapReduce for Multi-core and Multiprocessor Systems

HPCA '07 Proceedings of the 2007 IEEE 13th International Symposium on High Performance Computer Architecture
MapReduce: simplified data processing on large clusters

Communications of the ACM - 50th anniversary issue: 1958 - 2008
Aneka: Next-Generation Enterprise Grid Platform for e-Science and e-Business Applications

E-SCIENCE '07 Proceedings of the Third IEEE International Conference on e-Science and Grid Computing
Mars: a MapReduce framework on graphics processors

Proceedings of the 17th international conference on Parallel architectures and compilation techniques
CloudBLAST: Combining MapReduce and Virtualization on Distributed Resources for Bioinformatics Applications

ESCIENCE '08 Proceedings of the 2008 Fourth IEEE International Conference on eScience
A progressive multi-layer resource reconfiguration framework for time-shared grid systems

Future Generation Computer Systems
Supporting MapReduce on large-scale asymmetric multi-core clusters

ACM SIGOPS Operating Systems Review
Application Level Interoperability between Clouds and Grids

GPC '09 Proceedings of the 2009 Workshops at the Grid and Pervasive Computing Conference
CLOUDLET: towards mapreduce implementation on virtual machines

Proceedings of the 18th ACM international symposium on High performance distributed computing
MapReduce Programming Model for .NET-Based Cloud Computing

Euro-Par '09 Proceedings of the 15th International Euro-Par Conference on Parallel Processing
Improving MapReduce performance in heterogeneous environments

OSDI'08 Proceedings of the 8th USENIX conference on Operating systems design and implementation
On-demand data co-allocation with user-level cache for grids

Concurrency and Computation: Practice & Experience

Cloud MapReduce for Monte Carlo bootstrap applied to Metabolic Flux Analysis

Future Generation Computer Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents a grid-enabled MapReduce framework called ''Ussop''. Ussop provides its users with a set of C-language based MapReduce APIs and an efficient runtime system for exploiting the computing resources available on public-resource grids. Considering the volatility nature of the grid environment, Ussop introduces two novel task scheduling algorithms, namely, Variable-Sized Map Scheduling (VSMS) and Locality-Aware Reduce Scheduling (LARS). VSMS dynamically adjusts the size of map tasks according to the computing power of grid nodes. Moreover, LARS minimizes the data transfer cost of exchanging the intermediate data over a wide-area network. The experimental results indicate that both VSMS and LARS achieved superior performance than the conventional scheduling algorithms.