Parallel implementation of ant-based clustering algorithm based on hadoop

Authors:
Yan Yang;Xianhua Ni;Hongjun Wang;Yiteng Zhao
Affiliations:
School of Information Science and Technology, Southwest Jiaotong University, Chengdu, P.R. China,Key Lab of Cloud Computing and Intelligent Technology, Chengdu, Sichuan Province, P.R. China;School of Information Science and Technology, Southwest Jiaotong University, Chengdu, P.R. China,Key Lab of Cloud Computing and Intelligent Technology, Chengdu, Sichuan Province, P.R. China;School of Information Science and Technology, Southwest Jiaotong University, Chengdu, P.R. China,Key Lab of Cloud Computing and Intelligent Technology, Chengdu, Sichuan Province, P.R. China;School of Information Science and Technology, Southwest Jiaotong University, Chengdu, P.R. China
Venue:
ICSI'12 Proceedings of the Third international conference on Advances in Swarm Intelligence - Volume Part I
Year:
2012

Citing 3
Cited 0

The Google file system

SOSP '03 Proceedings of the nineteenth ACM symposium on Operating systems principles
MapReduce: simplified data processing on large clusters

OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Swarm intelligence: from natural to artificial systems [Book Reviews]

IEEE Transactions on Evolutionary Computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

Hadoop is a distributed system infrastructure of cloud computing. Based on the characteristics of ant-based clustering algorithm, the paper implements the parallelization of this algorithm using MapReduce on Hadoop. The Map function calculates the average similarity of the object with its neighborhood objects. The Reduce function processes the objects with the Map outputs and updates related information of both ants and the objects to get ready for the next job. Results on the Hadoop clusters show that our method can significantly improve the computational efficiency with the premise of maintaining clustering accuracy.