Maintaining views incrementally
SIGMOD '93 Proceedings of the 1993 ACM SIGMOD international conference on Management of data
View maintenance in a warehousing environment
SIGMOD '95 Proceedings of the 1995 ACM SIGMOD international conference on Management of data
Algorithms for deferred view maintenance
SIGMOD '96 Proceedings of the 1996 ACM SIGMOD international conference on Management of data
Supporting multiple view maintenance policies
SIGMOD '97 Proceedings of the 1997 ACM SIGMOD international conference on Management of data
Efficient view maintenance at data warehouses
SIGMOD '97 Proceedings of the 1997 ACM SIGMOD international conference on Management of data
Consistency Algorithms for Multi-Source Warehouse View Maintenance
Distributed and Parallel Databases - Special issue on parallel and distributed information systems
Shrinking the warehouse update Window
SIGMOD '99 Proceedings of the 1999 ACM SIGMOD international conference on Management of data
Evolvable view environment (EVE): non-equivalent view maintenance under schema changes
SIGMOD '99 Proceedings of the 1999 ACM SIGMOD international conference on Management of data
Proceedings of the 2nd ACM international workshop on Data warehousing and OLAP
How to roll a join: asynchronous incremental view maintenance
SIGMOD '00 Proceedings of the 2000 ACM SIGMOD international conference on Management of data
Maintaining data warehouses over changing information sources
Communications of the ACM
The Strobe algorithms for multi-source warehouse consistency
DIS '96 Proceedings of the fourth international conference on on Parallel and distributed information systems
Making views self-maintainable for data warehousing
DIS '96 Proceedings of the fourth international conference on on Parallel and distributed information systems
The EVE Approach: View Synchronization in Dynamic Distributed Environments
IEEE Transactions on Knowledge and Data Engineering
Data Integration using Self-Maintainable Views
EDBT '96 Proceedings of the 5th International Conference on Extending Database Technology: Advances in Database Technology
Multiple View Consistency for Data Warehousing
ICDE '97 Proceedings of the Thirteenth International Conference on Data Engineering
Concurrency Control Theory for Deferred Materialized Views
ICDT '97 Proceedings of the 6th International Conference on Database Theory
Incremental Maintenance of Materialized Views
DEXA '97 Proceedings of the 8th International Conference on Database and Expert Systems Applications
Implementing Incremental View Maintenance in Nested Data Models
DBLP-6 Proceedings of the 6th International Workshop on Database Programming Languages
The CVS Algorithm for View Synchronization in Evolvable Large-Scale Information Systems
EDBT '98 Proceedings of the 6th International Conference on Extending Database Technology: Advances in Database Technology
Maintaining Consistency in Partially Self-Maintainable Views at the Data Warehouse
DEXA '98 Proceedings of the 9th International Workshop on Database and Expert Systems Applications
The SDCC Framework For Integrating Existing Algorithms for Diverse Data Warehouse Maintenance Tasks
IDEAS '99 Proceedings of the 1999 International Symposium on Database Engineering & Applications
Multiversion-based view maintenance over distributed data sources
ACM Transactions on Database Systems (TODS)
Reducing the cost of accessing relations in incremental view maintenance
Decision Support Systems
Maintaining large update batches by restructuring and grouping
Information Systems
A user-driven data warehouse evolution approach for concurrent personalized analysis needs
Integrated Computer-Aided Engineering
DaWaK'05 Proceedings of the 7th international conference on Data Warehousing and Knowledge Discovery
Hi-index | 0.00 |
In a distributed environment, materialized views are used to integrate data from different information sources and then store them in some centralized location. In order to maintain such materialized views, maintenance queries need to be sent to information sources by the data warehouse management system. Due to the independence of the information sources and the data warehouse, concurrency issues are raised between the maintenance queries and the local update transactions at each information source. Recent solutions such as ECA and Strobe tackle such concurrent maintenance, however with the requirement of quiescence of the information sources. SWEEP and POSSE overcome this limitation by decomposing the global maintenance query into smaller subqueries to be sent to every information source and then performing conflict correction locally at the data warehouse. Note that all these previous approaches handle the data updates one at a time. Hence either some of the information sources or the data warehouse is likely to be idle during most of the maintenance process. In this paper, we propose that a set of updates should be maintained in parallel by several concurrent maintenance processes so that both the information sources as well as the warehouse would be utilized more fully throughout the maintenance process. This parallelism should then improve the overall maintenance performance. For this we have developed a parallel view maintenance algorithm, called PVM, that substantially improves upon the performance of previous maintenance approaches by handling a set of data updates at the same time. The parallel handling of a set of updates is orthogonal to the particular maintenance algorithm applied to the handling of each individual update. In order to perform parallel view maintenance, we have identified two critical issues that must be overcome: (1) detecting maintenance-concurrent data updates in a parallel mode and (2) correcting the problem that the data warehouse commit order may not correspond to the data warehouse update processing order due to parallel maintenance handling. In this work, we provide solutions to both issues. For the former, we insert a middle-layer timestamp assignment module for detecting maintenance-concurrent data updates without requiring any global clock synchronization. For the latter, we introduce the negative counter concept to solve the problem of variant orders of committing effects of data updates to the data warehouse. We provide a proof of the correctness of PVM that guarantees that our strategy indeed generates the correct final data warehouse state. We have implemented both SWEEP and PVM in our EVE data warehousing system. Our performance study demonstrates that a manyfold performance improvement is achieved by PVM over SWEEP.