Design of a data warehouse system for network/web services
Proceedings of the thirteenth ACM international conference on Information and knowledge management
Online Hierarchical Clustering in a Data Warehouse Environment
ICDM '05 Proceedings of the Fifth IEEE International Conference on Data Mining
Target-based database synchronization
Proceedings of the 2010 ACM Symposium on Applied Computing
Real-time data warehouse: a solution and evaluation
International Journal of Business Intelligence and Data Mining
| Hi-index | 0.00 |
This paper seeks to highlight an area important to commercial data warehouse deployments that has received limited research attention, namely, the extraction of changes to the data at the source systems. We refer to these changes as deltas. Extracting deltas from source systems is the first step in the incremental maintenance of data warehouses. A common assumption among current incremental maintenance methods is that deltas are somehow made available - normally in the form of differential files. Extraction of deltas from source systems is often not a straight forward process nor an efficient one.In this paper, we analyze how deltas can be extracted from large systems. We analyze delta extraction methods that are currently available, namely, time stamps, differential snapshots, triggers, and archive logs. We point out the strengths and weaknesses of each method through analysis and when appropriate through experimentation. We have been investigating the method called Op-Delta at Boeing that better suits delta extraction from large integrated systems. We discuss the benefits of Op-Delta, discuss how it could be implemented, and present comparative results from our experimentation.