Large-scale, high-resolution agricultural systems modeling using a hybrid approach combining grid computing and parallel processing

  • Authors:

  • Gang Zhao;Brett A. Bryan;Darran King;Zhongkui Luo;Enli Wang;Ulrike Bende-Michl;Xiaodong Song;Qiang Yu

  • Affiliations:

  • Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, 11A Datun Road, Anwai, Beijing 100101, China and CSIRO Ecosystem Sciences, Waite Campus, Urrbrae, SA 5 ...;CSIRO Ecosystem Sciences, Waite Campus, Urrbrae, SA 5064, Australia;CSIRO Ecosystem Sciences, Waite Campus, Urrbrae, SA 5064, Australia;CSIRO Land and Water, Black Mountain, Canberra, ACT 2601, Australia;CSIRO Land and Water, Black Mountain, Canberra, ACT 2601, Australia;CSIRO Land and Water, Black Mountain, Canberra, ACT 2601, Australia;CSIRO Ecosystem Sciences, Waite Campus, Urrbrae, SA 5064, Australia and Institute of Urban Environment, Chinese Academy of Sciences, Xiamen 361021, China;Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, 11A Datun Road, Anwai, Beijing 100101, China and Plant Functional Biology and Climate Change Cluster, ...

  • Venue:
  • Environmental Modelling & Software
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

The solution of complex global challenges in the land system, such as food and energy security, requires information on the management of agricultural systems at a high spatial and temporal resolution over continental or global extents. However, computing capacity remains a barrier to large-scale, high-resolution agricultural modeling. To model wheat production, soil carbon, and nitrogen dynamics in Australia's cropping regions at a high resolution, we developed a hybrid computing approach combining parallel processing and grid computing. The hybrid approach distributes tasks across a heterogeneous grid computing pool and fully utilizes all the resources of computers within the pool. We simulated 325 management scenarios (nitrogen application rates and stubble management) at a daily time step over 122 years, for 12,707 climate-soil zones using the Windows-based Agricultural Production Systems SIMulator (APSIM). These simulations would have taken over 30 years on a single computer. Our hybrid high performance computing (HPC) approach completed the modeling within 10.5 days-a speed-up of over 1000 times-with most jobs finishing within the first few days. The approach utilizes existing idle organization-wide computing resources and eliminates the need to translate Windows-based models to other operating systems for implementation on computing clusters. There are however, numerous computing challenges that need to be addressed for the effective use of these techniques and there remain several potential areas for further performance improvement. The results demonstrate the effectiveness of the approach in making high-resolution modeling of agricultural systems possible over continental and global scales.