Why you should care about quantile regression

  • Authors:

  • Augusto Born de Oliveira;Sebastian Fischmeister;Amer Diwan;Matthias Hauswirth;Peter F. Sweeney

  • Affiliations:

  • University of Waterloo, Waterloo, ON, Canada;University of Waterloo, Waterloo, ON, Canada;Google Inc., Mountain View, CA, USA;University of Lugano, Lugano, Switzerland;IBM T.J. Watson Research Center, Yorktown Heights, NY, USA

  • Venue:
  • Proceedings of the eighteenth international conference on Architectural support for programming languages and operating systems
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

Research has shown that correctly conducting and analysing computer performance experiments is difficult. This paper investigates what is necessary to conduct successful computer performance evaluation by attempting to repeat a prior experiment: the comparison between two Linux schedulers. In our efforts, we found that exploring an experimental space through a series of incremental experiments can be inconclusive, and there may be no indication of how much experimentation will be enough. Analysis of variance (ANOVA), a traditional analysis method, is able to partly solve the problems with the previous approach, but we demonstrate that ANOVA can be insufficient for proper analysis due to the requirements it imposes on the data. Finally, we demonstrate the successful application of quantile regression, a recent development in statistics, to computer performance experiments. Quantile regression can provide more insight into the experiment than ANOVA, with the additional benefit of being applicable to data from any distribution. This property makes it especially useful in our field, since non-normally distributed data is common in computer experiments.