SCAN: a structural clustering algorithm for networks
Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
PSCAN: A Parallel Structural Clustering Algorithm for Big Networks in MapReduce
AINA '13 Proceedings of the 2013 IEEE 27th International Conference on Advanced Information Networking and Applications
| Hi-index | 0.00 |
There is an increasing need for a storage system for petabyte scale graphs. In an attempt along the line, in this paper we develop a graph storage system, called Graph Store, for large graphs on top of the Hadoop Distributed File System (HDFS). Graph Store provides efficient graph storage and processing in a package. This paper also addresses critical challenges of Hadoop ecosystem to achieve the goals of Graph Store. One of them is a solution for "hotspots" in Hadoop and the solution proposed Congestion Prevention, Detection and Avoidance (CPDAB) algorithm. To demonstrate the performance of the Graph Store, a set of test cases are implemented to test on several synthetic and real network graphs including 41 million vertices user-follower network collected from Twitter in 2009. Our experiments on the proposed Graph Store found a superior performance.