Two-phase data warehouse optimized for data mining

Authors:
Balázs Rácz;Csaba István Sidló;András Lukács;András A. Benczúr
Affiliations:
Informatics Laboratory, Computer and Automation Research Institute of the Hungarian Academy of Sciences, Budapest, Hungary;Informatics Laboratory, Computer and Automation Research Institute of the Hungarian Academy of Sciences, Budapest, Hungary;Informatics Laboratory, Computer and Automation Research Institute of the Hungarian Academy of Sciences, Budapest, Hungary;Informatics Laboratory, Computer and Automation Research Institute of the Hungarian Academy of Sciences, Budapest, Hungary
Venue:
BIRTE'06 Proceedings of the 1st international conference on Business intelligence for the real-time enterprises
Year:
2006

Citing 21
Cited 0

Implementing data cubes efficiently

SIGMOD '96 Proceedings of the 1996 ACM SIGMOD international conference on Management of data
An overview of data warehousing and OLAP technology

ACM SIGMOD Record
Integrating association rule mining with relational database systems: alternatives and implications

SIGMOD '98 Proceedings of the 1998 ACM SIGMOD international conference on Management of data
Data mining: concepts and techniques

Data mining: concepts and techniques
Designing data marts for data warehouses

ACM Transactions on Software Engineering and Methodology (TOSEM)
Data Mining Your Website

Data Mining Your Website
Rise of the Network Society: The Information Age: Economy, Society and Culture

Rise of the Network Society: The Information Age: Economy, Society and Culture
Models and issues in data stream systems

Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Clickstream Data Warehousing

Clickstream Data Warehousing
Data Cube: A Relational Aggregation Operator Generalizing Group-By, Cross-Tab, and Sub-Totals

Data Mining and Knowledge Discovery
MSQL: A Query Language for Database Mining

Data Mining and Knowledge Discovery
On Using a Warehouse to Analyze Web Logs

Distributed and Parallel Databases
Integrating Data Mining with SQL Databases: OLE DB for Data Mining

Proceedings of the 17th International Conference on Data Engineering
Discovering Web Access Patterns and Trends by Applying OLAP and Data Mining Technology on Web Logs

ADL '98 Proceedings of the Advances in Digital Libraries Conference
Web usage mining: discovery and applications of usage patterns from Web data

ACM SIGKDD Explorations Newsletter
High Density Compression of Log Files

DCC '04 Proceedings of the Conference on Data Compression
Decision support queries on a tape-resident data warehouse

Information Systems
C-store: a column-oriented DBMS

VLDB '05 Proceedings of the 31st international conference on Very large data bases
Integrating compression and execution in column-oriented database systems

Proceedings of the 2006 ACM SIGMOD international conference on Management of data
Data compression in Oracle

VLDB '03 Proceedings of the 29th international conference on Very large data bases - Volume 29
Shaping SQL-Based frequent pattern mining algorithms

KDID'05 Proceedings of the 4th international conference on Knowledge Discovery in Inductive Databases

Quantified Score

Hi-index	0.00

Visualization

Abstract

We propose a new, heterogeneous data warehouse architecture where a first phase traditional relational OLAP warehouse coexist with a second phase data in compressed form optimized for data mining. Aggregations and metadata for the entire time frame are stored in the first phase relational database. The main advantage of the second phase is its reduced I/O requirement that enables very high throughput processing by sequential read-only data stream algorithms. It becomes feasible to run speed optimized queries and data mining operations on the entire time frame of most granular data. The second phase also enables long term data storage and analysis using a very efficient compressed format at low storage costs even for historical data. The proposed architecture fits existing data warehouse solutions. We show the effectiveness of the two-phase data warehouse through a case study of a large web portal.