MapReduce: simplified data processing on large clusters
OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Parallel Skeletons for Variable-Length Lists in SkeTo Skeleton Library
Euro-Par '09 Proceedings of the 15th International Euro-Par Conference on Parallel Processing
Skandium: Multi-core Programming with Algorithmic Skeletons
PDP '10 Proceedings of the 2010 18th Euromicro Conference on Parallel, Distributed and Network-based Processing
Load Balancing Algorithms with Partial Information Management for the DLML Library
PDP '10 Proceedings of the 2010 18th Euromicro Conference on Parallel, Distributed and Network-based Processing
Beginning OpenVPN 2.0.9
EuroPar'10 Proceedings of the 16th international Euro-Par conference on Parallel processing: Part I
| Hi-index | 0.00 |
The Data List Management Library (DLML) processes data lists in parallel, balancing the workload transparently to programmers. Programmers only need to organise data into a list, use DLML functions to insert and get data items, and specify the sequential function(s) to process each data item according to the application logic. The first design of DLML was targeted for use at a single cluster. This paper presents DLML-Grid, a software architecture for DLML to run in Grid environments composed of multiple distributed clusters. The architecture is hierarchical and tends to localise communication within clusters, thus reducing communication overhead. Using OpenVPN, we implemented a prototype version of DLML-Grid to gather empirical results on its performance using two clusters and two applications whose workload is static and dynamically generated. DLML-Grid performs much better than DLML overall.