A Methodology for Evaluating Predictive Metrics

  • Authors:
  • Jarrett Rosenberg

  • Affiliations:
  • -

  • Venue:
  • METRICS '98 Proceedings of the 5th International Symposium on Software Metrics
  • Year:
  • 1998

Quantified Score

Hi-index 0.00

Visualization

Abstract

A central question in software metrics is how well one or more metrics predict some measure of interest, such as defect density or development effort. Numerous proposals for new metrics have been made over the past decades, some with attempts at empirical evaluation. Unfortunately, most evaluations of metrics are crude at best, and often completely invalid. This paper describes an accepted approach to the evaluation of the predictive ability of one or more metrics, using well-established statistical methods. Such topics as the importance of examining data prior to analysis, avoidance of model violations such as collinearity, and the proper use of univariate and multivariate techniques are discussed and illustrated, as well as common mistakes such as the use of automated procedures like stepwise regression. More advanced statistical techniques such as logistic regression and signal detection theory are briefly discussed.