Optimizing disk storage to support statistical analysis operations

  • Authors:
  • Allen Parrish;Susan Vrbsky;Brandon Dixon;Weigang Ni

  • Affiliations:
  • Department of Computer Science, The University of Alabama, Box 870290, Tuscaloos, AL;Department of Computer Science, The University of Alabama, Box 870290, Tuscaloos, AL;Department of Computer Science, The University of Alabama, Box 870290, Tuscaloos, AL;Department of Computer Science, The University of Alabama, Box 870290, Tuscaloos, AL

  • Venue:
  • Decision Support Systems
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

Data stored in spreadsheets and relational database tables can be viewed as "worksheets" consisting of rows and columns, with rows corresponding to records. Correspondingly, the typical practice is to store the data on disk in row major order. While this practice is reasonable in many cases, it is not necessarily the best practice when computation is dominated by column-based statistics. This short note discusses the performance tradeoffs between row major and column major storage of data in the context of statistical data analysis. A comparison of a software package utilizing column major storage and one using row major storage confirms our results.