The equi-join processing and optimization on ring architecture key/value database

Authors:
Xite Wang;Derong Shen;Tiezheng Nie;Yue Kou;Ge Yu
Affiliations:
Key Laboratory of Medical Image Computing (NEU), Ministry of Education, College of Information Science and Engineering, Northeastern University, P.R. China;Key Laboratory of Medical Image Computing (NEU), Ministry of Education, College of Information Science and Engineering, Northeastern University, P.R. China;Key Laboratory of Medical Image Computing (NEU), Ministry of Education, College of Information Science and Engineering, Northeastern University, P.R. China;Key Laboratory of Medical Image Computing (NEU), Ministry of Education, College of Information Science and Engineering, Northeastern University, P.R. China;Key Laboratory of Medical Image Computing (NEU), Ministry of Education, College of Information Science and Engineering, Northeastern University, P.R. China
Venue:
APWeb'12 Proceedings of the 14th Asia-Pacific international conference on Web Technologies and Applications
Year:
2012

Citing 6
Cited 0

Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web

STOC '97 Proceedings of the twenty-ninth annual ACM symposium on Theory of computing
Dynamo: amazon's highly available key-value store

Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles
Cassandra: a decentralized structured storage system

ACM SIGOPS Operating Systems Review
Processing theta-joins using MapReduce

Proceedings of the 2011 ACM SIGMOD International Conference on Management of data
Llama: leveraging columnar storage for scalable join processing in the MapReduce framework

Proceedings of the 2011 ACM SIGMOD International Conference on Management of data
Efficient processing of data warehousing queries in a split execution environment

Proceedings of the 2011 ACM SIGMOD International Conference on Management of data

Quantified Score

Hi-index	0.00

Visualization

Abstract

Huge data analysis (e.g. equi-join) on key/value data storage system is a very novel and necessary issue. For the master-slave architecture distributed system(MSADS) (e.g. Hadoop), equi-join can be implemented by utilizing MapReduce framework which is designed for performing scalable parallel data analysis on MSADS. However, for the ring architecture distributed system (RADS), there has no general method for processing data analysis task (e.g. equi-join). Hence in this paper, a novel approach is proposed for processing equi-join on RADS. Firstly, by making in-depth analysis of RADS, we propose a new type of index, ColumnValue index(CVI), based on ColumnFamily data model. Then, by utilizing CVI, an efficient algorithm, called pre-join table generation algorithm (PJTG), is proposed to process the equi-join query on RADS, and a memory index(MI) is utilized to further improve the performance of PJTG. In addition, the update method for the join result is present , and the update is caused by the alteration of original data. Finally, the validity of PJTG is verified through plenty of simulation experiments. Experimental results show that the proposed algorithm is an effective way to solve the equi-join query problem on RADS and could meet the requirements of practical applications.