CineGrid Exchange: A workflow-based peta-scale distributed storage platform on a high-speed network
Future Generation Computer Systems
| Hi-index | 0.00 |
We present a novel rails approach so that future e-Science applications can effectively exploit future system architectures, including multi-core and many-core architectures, multiple network cards, multiple graphical processing units and hybrid hierarchical memory architectures. We define "rail" as the co-scheduling of two or more of these resources. This approach enables creation of parallel multi-rails through every aspect of an end system: from processing on the multi- and many cores, to generation of multiple data flows and streaming over multi-lane network interface card (NIC) connected via a parallel interconnect. We describe a novel open-source multi-rail toolkit and the evaluation of end-system parameters that impact the efficiency of such multi-rail systems, including Interrupt, Memory, Thread and Core Affinities -- key properties for achieving scalable performance.