The R*-tree: an efficient and robust access method for points and rectangles
SIGMOD '90 Proceedings of the 1990 ACM SIGMOD international conference on Management of data
Parallel database systems: the future of high performance database systems
Communications of the ACM
T2: a customizable parallel database for multi-dimensional data
ACM SIGMOD Record
The pyramid-technique: towards breaking the curse of dimensionality
SIGMOD '98 Proceedings of the 1998 ACM SIGMOD international conference on Management of data
The multidimensional database system RasDaMan
SIGMOD '98 Proceedings of the 1998 ACM SIGMOD international conference on Management of data
Multidimensional binary search trees used for associative searching
Communications of the ACM
The K-D-B-tree: a search structure for large multidimensional dynamic indexes
SIGMOD '81 Proceedings of the 1981 ACM SIGMOD international conference on Management of data
R-trees: a dynamic index structure for spatial searching
SIGMOD '84 Proceedings of the 1984 ACM SIGMOD international conference on Management of data
Multidimensional Database Technology
Computer
Scalability Analysis of Declustering Methods for Multidimensional Range Queries
IEEE Transactions on Knowledge and Data Engineering
Titan: A High-Performance Remote Sensing Database
ICDE '97 Proceedings of the Thirteenth International Conference on Data Engineering
Efficient Organization of Large Multidimensional Arrays
Proceedings of the Tenth International Conference on Data Engineering
VLDB '94 Proceedings of the 20th International Conference on Very Large Data Bases
Physical Schemas for Large Multidimensional Arrays in Scientific Computing Applications
Proceedings of the Seventh International Working Conference on Scientific and Statistical Database Management
DEXA '02 Proceedings of the 13th International Conference on Database and Expert Systems Applications
Query processing techniques for arrays
The VLDB Journal — The International Journal on Very Large Data Bases
Storage of Multidimensional Arrays Based on Arbitrary Tiling
ICDE '99 Proceedings of the 15th International Conference on Data Engineering
The Priority R-tree: a practically efficient and worst-case optimal R-tree
SIGMOD '04 Proceedings of the 2004 ACM SIGMOD international conference on Management of data
Efficient High-Dimensional Indexing by Superimposing Space-Partitioning Schemes
IDEAS '04 Proceedings of the International Database Engineering and Applications Symposium
An extendible multidimensional array system for MOLAP
Proceedings of the 2006 ACM symposium on Applied computing
Optimal chunking of large multidimensional arrays for data warehousing
Proceedings of the ACM tenth international workshop on Data warehousing and OLAP
MAD skills: new analysis practices for big data
Proceedings of the VLDB Endowment
Skew-resistant parallel processing of feature-extracting scientific user-defined functions
Proceedings of the 1st ACM symposium on Cloud computing
Overview of sciDB: large scale array storage, processing and analysis
Proceedings of the 2010 ACM SIGMOD International Conference on Management of data
Scalable clustering algorithm for N-body simulations in a shared-nothing cluster
SSDBM'10 Proceedings of the 22nd international conference on Scientific and statistical database management
Hybrid merge/overlap execution technique for parallel array processing
Proceedings of the EDBT/ICDT 2011 Workshop on Array Databases
Distribution rules for array database queries
DEXA'05 Proceedings of the 16th international conference on Database and Expert Systems Applications
Hybrid merge/overlap execution technique for parallel array processing
Proceedings of the EDBT/ICDT 2011 Workshop on Array Databases
SSDBM'11 Proceedings of the 23rd international conference on Scientific and statistical database management
SciHadoop: array-based query processing in Hadoop
Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis
Towards scalable array-oriented active storage: the pyramid approach
ACM SIGOPS Operating Systems Review
Intel "big data" science and technology center vision and execution plan
ACM SIGMOD Record
Proceedings of the 25th International Conference on Scientific and Statistical Database Management
Astronomical data processing in EXTASCID
Proceedings of the 25th International Conference on Scientific and Statistical Database Management
SIDR: structure-aware intelligent data routing in Hadoop
SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Can we analyze big data inside a DBMS?
Proceedings of the sixteenth international workshop on Data warehousing and OLAP
| Hi-index | 0.00 |
We present the design, implementation, and evaluation of ArrayStore, a new storage manager for complex, parallel array processing. ArrayStore builds on prior work in the area of multidimensional data storage, but considers the new problem of supporting a parallel and more varied workload comprising not only range-queries, but also binary operations such as joins and complex user-defined functions. This paper makes two key contributions. First, it examines several existing single-site storage management strategies and array partitioning strategies to identify which combination is best suited for the array-processing workload above. Second, it develops a new and efficient storage-management mechanism that enables parallel processing of operations that must access data from adjacent partitions. We evaluate ArrayStore on over 80GB of real data from two scientific domains and real operators used in these domains. We show that ArrayStore outperforms previously proposed storage management strategies in the context of its diverse target workload.