An optimal relationship-based partitioning of large datasets

Authors:
Darko Capko;Aleksandar Erdeljan;Miroslav Popovic;Goran Svenda
Affiliations:
Faculty of Technical Sciences, Novi Sad, Serbia;Faculty of Technical Sciences, Novi Sad, Serbia;Faculty of Technical Sciences, Novi Sad, Serbia;Faculty of Technical Sciences, Novi Sad, Serbia
Venue:
ADBIS'10 Proceedings of the 14th east European conference on Advances in databases and information systems
Year:
2010

Citing 3
Cited 0

Multilevel diffusion schemes for repartitioning of adaptive meshes

Journal of Parallel and Distributed Computing - Special issue on dynamic load balancing
Genetic Algorithm and Graph Partitioning

IEEE Transactions on Computers
Particle swarm optimization for integer programming

CEC '02 Proceedings of the Evolutionary Computation on 2002. CEC '02. Proceedings of the 2002 Congress - Volume 02

Quantified Score

Hi-index	0.00

Visualization

Abstract

Modern adaptive applications utilize multiprocessor systems for efficient processing of large datasets where initial and dynamic partitioning of large datasets is necessary to obtain an optimal load balancing among processors. We applied evolutionary algorithms (Genetic Algorithm and Particle Swarm Optimization) for initial partitioning, and diffusion (DR) and cut-and-paste (CP) algorithms for dynamic partitioning. Modified versions of DR and CP algorithms are developed to improve dynamic partitioning running in NUMA multiprocessor systems. The proposed algorithms were applied on datasets describing large electricity power distribution systems and experimental results prove reductions of processor load imbalance and performance improvements.