Transaction reordering and grouping for continuous data loading

  • Authors:
  • Gang Luo;Jeffrey F. Naughton;Curt J. Ellmann;Michael W. Watzke

  • Affiliations:
  • IBM T.J. Watson Research Center;University of Wisconsin-Madison;University of Wisconsin-Madison;NCR

  • Venue:
  • BIRTE'06 Proceedings of the 1st international conference on Business intelligence for the real-time enterprises
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

With the increasing popularity of operational data warehousing, the ability to load data quickly and continuously into an RDBMS is becoming more and more important. However, in the presence of materialized join views, loading data concurrently into multiple base relations of the same materialized join view can cause a severe deadlock problem. To solve this problem, we propose reordering the data to be loaded so that at any time, for any materialized join view, data is only loaded into one of its base relations. Also, for load transactions on the relations that contain "aggregate" attributes, we propose using pre-aggregation to reduce the number of SQL statements in the load transactions. The advantages of our methods are demonstrated through experiments with a commercial parallel RDBMS.