The Accuracy of Fault Prediction in Modified Code - Statistical Model vs. Expert Estimation

  • Authors:

  • Piotr Tomaszewski;Jim Hakansson;Lars Lundberg;Hakan Grahn

  • Affiliations:

  • Blekinge Institute of Technology;Blekinge Institute of Technology;Blekinge Institute of Technology;Blekinge Institute of Technology

  • Venue:
  • ECBS '06 Proceedings of the 13th Annual IEEE International Symposium and Workshop on Engineering of Computer Based Systems
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

Fault prediction models still seem to be more popular in academia than in industry. In industry expert estimations of fault proneness are the most popular methods of deciding where to focus the fault detection efforts. In this paper we present a study in which we empirically evaluate the accuracy of fault prediction offered by statistical models as compared to expert estimations. The study is industry based. It involves a large telecommunication system and experts that were involved in the development of this system. Expert estimations are compared to simple prediction models built on another large system, also from the telecommunication domain. We show that the statistical methods clearly outperform the expert estimations. As the main reason for the superiority of the statistical models we see their ability to cope with large datasets, which results in their ability to perform reliable predictions for larger number of components in the system, as well as the ability to perform prediction at a more fine-grain level, e.g., at the class instead of at the component level.