SIGMOD '91 Proceedings of the 1991 ACM SIGMOD international conference on Management of data
Building an object-oriented database system: the story of 02
Building an object-oriented database system: the story of 02
Shoring up persistent applications
SIGMOD '94 Proceedings of the 1994 ACM SIGMOD international conference on Management of data
A framework for supporting data integration using the materialized and virtual approaches
SIGMOD '96 Proceedings of the 1996 ACM SIGMOD international conference on Management of data
Transforming databases with recursive data structures
Transforming databases with recursive data structures
Converting relational to object-oriented databases
ACM SIGMOD Record
The object database standard: ODMG 2.0
The object database standard: ODMG 2.0
Algorithms for loading object databases
Algorithms for loading object databases
Your mediators need data conversion!
SIGMOD '98 Proceedings of the 1998 ACM SIGMOD international conference on Management of data
Catching the boat with Strudel: experiences with a Web-site management system
SIGMOD '98 Proceedings of the 1998 ACM SIGMOD international conference on Management of data
Efficient resumption of interrupted warehouse loads
SIGMOD '00 Proceedings of the 2000 ACM SIGMOD international conference on Management of data
Efficient and extensible algorithms for multi query optimization
SIGMOD '00 Proceedings of the 2000 ACM SIGMOD international conference on Management of data
A View Mechanism for Object-Oriented Databases
EDBT '92 Proceedings of the 3rd International Conference on Extending Database Technology: Advances in Database Technology
WOL: A Language for Database Transformations and Constraints
ICDE '97 Proceedings of the Thirteenth International Conference on Data Engineering
Object Exchange Across Heterogeneous Information Sources
ICDE '95 Proceedings of the Eleventh International Conference on Data Engineering
A Generic Approach to Bulk Loading Multidimensional Index Structures
VLDB '97 Proceedings of the 23rd International Conference on Very Large Data Bases
Bulk-Loading Techniques for Object Databases and an Application to Relational Data
VLDB '98 Proceedings of the 24rd International Conference on Very Large Data Bases
Using Schema Matching to Simplify Heterogeneous Data Translation
VLDB '98 Proceedings of the 24rd International Conference on Very Large Data Bases
Multiview: A Methodology for Supporting Multiple Views in Object-Oriented Databases
VLDB '92 Proceedings of the 18th International Conference on Very Large Data Bases
OODB Bulk Loading Revisited: The Partitioned-List Approach
VLDB '95 Proceedings of the 21th International Conference on Very Large Data Bases
Nested Queries in Object Bases
DBLP-4 Proceedings of the Fourth International Workshop on Database Programming Languages - Object Models and Languages
Towards heterogeneous multimedia information systems: the Garlic approach
RIDE '95 Proceedings of the 5th International Workshop on Research Issues in Data Engineering-Distributed Object Management (RIDE-DOM'95)
Scaling heterogeneous databases and the design of Disco
ICDCS '96 Proceedings of the 16th International Conference on Distributed Computing Systems (ICDCS '96)
ICDE '02 Proceedings of the 18th International Conference on Data Engineering
Optimized Data Loading for a Multi-Terabyte Sky Survey Repository
SC '05 Proceedings of the 2005 ACM/IEEE conference on Supercomputing
Bulk loading a linear hash file
DaWaK'06 Proceedings of the 8th international conference on Data Warehousing and Knowledge Discovery
Research and implement of real-time data loading system IMIL
WISE'06 Proceedings of the 7th international conference on Web Information Systems
Hi-index | 0.00 |
Applications, such as warehouse maintenance, need to load large data volumes regularly. The efficiency of loading depends on the resources that are available at the source and at the target systems. Our work aims to understand the performance criteria that are involved in bulk loading data into a database and to devise tailored optimization strategies.Unlike commercial systems and previous research on the same topic, our approach follows the fundamental database principle of physical-logical independence. A loading program is represented as a sequence of algebraic expressions. This abstraction enables the use of appropriate algebraic rewritings to optimize a loading program and of a cost model that takes into consideration efficiency criteria such as the processing times at the source and target systems and the bandwidth between them. A slow-loading program may be preferable if it does not slow down other applications by consuming too much memory. Thus, we view the problem of optimizing a loading program as finding a compromise between several efficiency criteria.The ability to represent loading programs in an algebra and performance criteria in a cost model has two very desirable properties: reusability and efficiency. Database programmers do not have to write loading programs by hand. In addition, tuning loading programs becomes easier since programmers have a better control on the performance criteria specified in the cost model. The algebra captures data transformations that would have been otherwise hardcoded in loading programs. Consequently, richer optimizations can be explored. Finally, our optimization techniques are not specific to one particular system. They can be used for loading data and from to any structured store (e.g., relational, structured files).We implemented our ideas in a complete environment for migrating ODBC-compliant databases into the O2 object-oriented database system. This prototype provides a declarative view language to specify loading, an interface to specify directives, such as desired database physical organization and constraints on several criteria, such as resource and bandwidth consumption, an algebraic optimizer, a code generator, and an execution environment to control failures and guarantee incremental loading. Our experiments show that a tailored optimization is necessary when loading large data volumes into a database.