Accelerating Spatial Data Processing with MapReduce

Authors:
Kai Wang;Jizhong Han;Bibo Tu;Jiao Dai;Wei Zhou;Xuan Song
Affiliations:
-;-;-;-;-;-
Venue:
ICPADS '10 Proceedings of the 2010 IEEE 16th International Conference on Parallel and Distributed Systems
Year:
2010

Citing 0
Cited 4

A parallel input-output system for resolving spatial data challenges: an agent-based model case study

Proceedings of the ACM SIGSPATIAL Second International Workshop on High Performance and Distributed Geographic Information Systems
Collaborative geospatial feature search

Proceedings of the 20th International Conference on Advances in Geographic Information Systems
Processing multi-way spatial joins on map-reduce

Proceedings of the 16th International Conference on Extending Database Technology
CG_Hadoop: computational geometry in MapReduce

Proceedings of the 21st ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

Map Reduce is a key-value based programming model and an associated implementation for processing large data sets. It has been adopted in various scenarios and seems promising. However, when spatial computation is expressed straightforward by this key-value based model, difficulties arise due to unfit features and performance degradation. In this paper, we present methods as follows: 1) a splitting method for balancing workload, 2) pending file structure and redundant data partition dealing with relation between spatial objects, 3) a strip-based two-direction plane sweeping algorithm for computation accelerating. Based on these methods, ANN(All nearest neighbors) query and astronomical cross-certification are developed. Performance evaluation shows that the Map Reduce-based spatial applications outperform the traditional one on DBMS.