Cloud MapReduce for Monte Carlo bootstrap applied to Metabolic Flux Analysis

  • Authors:

  • Tolga Dalman;Tim DöRnemann;Ernst Juhnke;Michael Weitzel;Wolfgang Wiechert;Katharina NöH;Bernd Freisleben

  • Affiliations:

  • Institute of Bio- and Geosciences 1: Biotechnology 2, Forschungszentrum Jülich, Wilhelm-Johnen-Straíe, D-52428 Jülich, Germany;Department of Mathematics & Computer Science and Center for Synthetic Microbiology, University of Marburg, Hans-Meerwein-Straííe 3, D-35032 Marburg, Germany;Department of Mathematics & Computer Science and Center for Synthetic Microbiology, University of Marburg, Hans-Meerwein-Straííe 3, D-35032 Marburg, Germany;Institute of Bio- and Geosciences 1: Biotechnology 2, Forschungszentrum Jülich, Wilhelm-Johnen-Straíe, D-52428 Jülich, Germany;Institute of Bio- and Geosciences 1: Biotechnology 2, Forschungszentrum Jülich, Wilhelm-Johnen-Straíe, D-52428 Jülich, Germany;Institute of Bio- and Geosciences 1: Biotechnology 2, Forschungszentrum Jülich, Wilhelm-Johnen-Straíe, D-52428 Jülich, Germany;Department of Mathematics & Computer Science and Center for Synthetic Microbiology, University of Marburg, Hans-Meerwein-Straííe 3, D-35032 Marburg, Germany

  • Venue:
  • Future Generation Computer Systems
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

The MapReduce architectural pattern popularized by Google has successfully been utilized in several scientific applications. Up until now, MapReduce is rarely employed in the field of Systems Biology. We investigate whether a MapReduce approach utilizing on-demand resources from a Cloud is suitable to perform simulation tasks in the area of Metabolic Flux Analysis (MFA). An Amazon ElasticMapReduce Cloud implementation of the parallel, parametric Monte Carlo bootstrap in the context to ^1^3C-MFA is presented. The seamless integration of the application into a service-oriented, BPEL-based scientific workflow framework is shown. A comparison of a straightforward MapReduce implementation using the Hadoop streaming interface on various Amazon ElasticMapReduce instance types and a single CPU core computation approach reveals a speedup of 17 on 64 Amazon cores. I/O operations on many small files within the Reduce step were identified as the limiting step. By exploiting the Hadoop Java API, making use of built-in data types and tuning problem-specific Hadoop parameters, the I/O issues could be resolved. With the revised implementation, a speedup of up to 48 could be achieved on 64 Amazon cores. To investigate the runtimes of a realistic ^1^3C-MFA analysis, 50,000 Monte Carlo samples with a typical metabolic network model have been performed on 20 virtual nodes in 24 h and 23 min with a total cost of $384. Our work demonstrates the possibility to perform scalable Systems Biology applications using Amazon's Cloud MapReduce service.