Variable-Sized map and locality-aware reduce on public-resource grids

Authors:
Po-Cheng Chen;Yen-Liang Su;Jyh-Biau Chang;Ce-Kuen Shieh
Affiliations:
Institute of Computer and Communication Engineering, Department of Electrical Engineering, National Cheng Kung University, Taiwan;Institute of Computer and Communication Engineering, Department of Electrical Engineering, National Cheng Kung University, Taiwan;Department of Digital Applications, Leader University, Taiwan;Institute of Computer and Communication Engineering, Department of Electrical Engineering, National Cheng Kung University, Taiwan
Venue:
GPC'10 Proceedings of the 5th international conference on Advances in Grid and Pervasive Computing
Year:
2010

Citing 10
Cited 0

Teamster-G: a grid-enabled software DSM system

CCGRID '05 Proceedings of the Fifth IEEE International Symposium on Cluster Computing and the Grid (CCGrid'05) - Volume 2 - Volume 02
MapReduce: simplified data processing on large clusters

OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Evaluating MapReduce for Multi-core and Multiprocessor Systems

HPCA '07 Proceedings of the 2007 IEEE 13th International Symposium on High Performance Computer Architecture
Mars: a MapReduce framework on graphics processors

Proceedings of the 17th international conference on Parallel architectures and compilation techniques
A progressive multi-layer resource reconfiguration framework for time-shared grid systems

Future Generation Computer Systems
Supporting MapReduce on large-scale asymmetric multi-core clusters

ACM SIGOPS Operating Systems Review
Application Level Interoperability between Clouds and Grids

GPC '09 Proceedings of the 2009 Workshops at the Grid and Pervasive Computing Conference
CLOUDLET: towards mapreduce implementation on virtual machines

Proceedings of the 18th ACM international symposium on High performance distributed computing
MapReduce Programming Model for .NET-Based Cloud Computing

Euro-Par '09 Proceedings of the 15th International Euro-Par Conference on Parallel Processing
Improving MapReduce performance in heterogeneous environments

OSDI'08 Proceedings of the 8th USENIX conference on Operating systems design and implementation

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents a grid-enabled MapReduce framework called “Ussop” Ussop provides its users with a set of C-language based MapReduce APIs and an efficient runtime system for exploiting the computing resources available on public-resource grids Considering the volatility nature of the grid environment, Ussop introduces two novel task scheduling algorithms, namely: Variable-Sized Map Scheduling (VSMS) and Locality-Aware Reduce Scheduling (LARS) VSMS dynamically adjusts the size of the map tasks according to the computing power of grid nodes Moreover, LARS minimizes the data transfer cost of exchanging the intermediate data over a wide-area network The experimental results indicate that both VSMS and LARS achieved superior performance than the conventional scheduling algorithms.