Communications of the ACM
Speeding up bulk-loading of quadtrees
GIS '97 Proceedings of the 5th ACM international workshop on Advances in geographic information systems
Efficient resumption of interrupted warehouse loads
SIGMOD '00 Proceedings of the 2000 ACM SIGMOD international conference on Management of data
Incremental Organization for Data Recording and Warehousing
VLDB '97 Proceedings of the 23rd International Conference on Very Large Data Bases
Weaving Relations for Cache Performance
Proceedings of the 27th International Conference on Very Large Data Bases
OODB Bulk Loading Revisited: The Partitioned-List Approach
VLDB '95 Proceedings of the 21th International Conference on Very Large Data Bases
Efficient Bulk Loading of Large High-Dimensional Indexes
DaWaK '99 Proceedings of the First International Conference on Data Warehousing and Knowledge Discovery
Bulk Loading a Data Warehouse Built Upon a UB-Tree
IDEAS '00 Proceedings of the 2000 International Symposium on Database Engineering & Applications
Queue - Storage
A declarative approach to optimize bulk loading into databases
ACM Transactions on Database Systems (TODS)
| Hi-index | 0.00 |
We study the problem of bulk loading a linear hash file; the problem is that a good hash function is able to distribute records into random locations in the file; however, performing a random disk access for each record can be costly and this cost increases with the size of the file. We propose a bulk loading algorithm that can avoid random disk accesses by reducing multiple accesses to the same location into a single access and reordering the accesses such that the pages are accessed sequentially. Our analysis shows that our algorithm is near-optimal with a cost roughly equal to the cost of sorting the dataset, thus the algorithm can scale up to very large datasets. Our experiments show that our method can improve upon the Berkeley DB load utility, in terms of running time, by two orders of magnitude and the improvements scale up well with the size of the dataset.