Evaluating the performance and scalability of mapreduce applications on X10

Authors:
Chao Zhang;Chenning Xie;Zhiwei Xiao;Haibo Chen
Affiliations:
Parallel Processing Institute, Fudan University;Parallel Processing Institute, Fudan University;Parallel Processing Institute, Fudan University;Parallel Processing Institute, Fudan University
Venue:
APPT'11 Proceedings of the 9th international conference on Advanced parallel processing technologies
Year:
2011

Citing 11
Cited 3

X10: an object-oriented approach to non-uniform cluster computing

OOPSLA '05 Proceedings of the 20th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications
X10: concurrent programming for modern architectures

Proceedings of the 12th ACM SIGPLAN symposium on Principles and practice of parallel programming
Deadlock-free scheduling of X10 computations with bounded resources

Proceedings of the nineteenth annual ACM symposium on Parallel algorithms and architectures
MapReduce: simplified data processing on large clusters

Communications of the ACM - 50th anniversary issue: 1958 - 2008
Parallel computing with x10

Proceedings of the 1st international workshop on Multicore software engineering
Static Detection of Place Locality and Elimination of Runtime Checks

APLAS '08 Proceedings of the 6th Asian Symposium on Programming Languages and Systems
Efficient, portable implementation of asynchronous multi-place programs

Proceedings of the 14th ACM SIGPLAN symposium on Principles and practice of parallel programming
Reducing task creation and termination overhead in explicitly parallel programs

Proceedings of the 19th international conference on Parallel architectures and compilation techniques
Lifeline-based global load balancing

Proceedings of the 16th ACM symposium on Principles and practice of parallel programming
Efficient optimization of memory accesses in parallel programs

Efficient optimization of memory accesses in parallel programs
Hierarchical place trees: a portable abstraction for task parallelism and data movement

LCPC'09 Proceedings of the 22nd international conference on Languages and Compilers for Parallel Computing

Characterization of Smith-Waterman sequence database search in X10

Proceedings of the 2012 ACM SIGPLAN X10 Workshop
X10-FT: transparent fault tolerance for APGAS language and runtime

Proceedings of the 2013 International Workshop on Programming Models and Applications for Multicores and Manycores
X10-FT: Transparent fault tolerance for APGAS language and runtime

Parallel Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

MapReduce has been shown to be a simple and efficient way to harness the massive resources of clusters. Recently, researchers propose using partitioned global address space (PGAS) based language and runtime to ease the programming of large-scale clusters. In this paper, we present an empirical study on the effectiveness of running MapReduce applications on a typical PGAS language runtime called X10. By tuning the performance of two applications on X10 platforms, we successfully eliminate several performance bottlenecks related to I/O processing. We also identify several remaining problems and propose several approaches to remedying them. Our final performance evaluation on a small-scale multicore cluster shows that the MapReduce applications written with X10 notably outperform those in Hadoop in most cases. Detailed analysis reveals that the major performance advantages come from a simplified task management and data storage scheme.