An overview of data warehousing and OLAP technology
ACM SIGMOD Record
Shrinking the warehouse update Window
SIGMOD '99 Proceedings of the 1999 ACM SIGMOD international conference on Management of data
Lazy Aggregates for Real-Time OLAP
DaWaK '99 Proceedings of the First International Conference on Data Warehousing and Knowledge Discovery
Scientific data management in the coming decade
ACM SIGMOD Record
Towards generating ETL processes for incremental loading
IDEAS '08 Proceedings of the 2008 international symposium on Database engineering & applications
Data integration flows for business intelligence
Proceedings of the 12th International Conference on Extending Database Technology: Advances in Database Technology
An architecture for recycling intermediates in a column-store
Proceedings of the 2009 ACM SIGMOD International Conference on Management of data
Fundamentals of Data Warehouses
Fundamentals of Data Warehouses
SciQL: bridging the gap between science and relational DBMS
Proceedings of the 15th Symposium on International Database Engineering & Applications
NoDB: efficient query execution on raw data files
SIGMOD '12 Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data
Hi-index | 0.00 |
Both scientific data and business data have analytical needs. Analysis takes place after a scientific data warehouse is eagerly filled with all data from external data sources (repositories). This is similar to the initial loading stage of Extract, Transform, and Load (ETL) processes that drive business intelligence. ETL can also help scientific data analysis. However, the initial loading is a time and resource consuming operation. It might not be entirely necessary, e.g. if the user is interested in only a subset of the data. We propose to demonstrate Lazy ETL, a technique to lower costs for initial loading. With it, ETL is integrated into the query processing of the scientific data warehouse. For a query, only the required data items are extracted, transformed, and loaded transparently on-the-fly. The demo is built around concrete implementations of Lazy ETL for seismic data analysis. The seismic data warehouse is ready for query processing, without waiting for long initial loading. The audience fires analytical queries to observe the internal mechanisms and modifications that realize each of the steps; lazy extraction, transformation, and loading.