How to measure success of fault prediction models

Authors:
Thomas J. Ostrand;Elaine J. Weyuker
Affiliations:
AT&T Labs - Research, Florham Park, NJ;AT&T Labs - Research, Florham Park, NJ
Venue:
Fourth international workshop on Software quality assurance: in conjunction with the 6th ESEC/FSE joint meeting
Year:
2007

Citing 8
Cited 13

The distribution of faults in a large industrial software system

ISSTA '02 Proceedings of the 2002 ACM SIGSOFT international symposium on Software testing and analysis
An empirical evaluation of fault-proneness models

Proceedings of the 24th International Conference on Software Engineering
Early Quality Prediction: A Case Study in Telecommunications

IEEE Software
Predicting the Location and Number of Faults in Large Software Systems

IEEE Transactions on Software Engineering
Looking for bugs in all the right places

Proceedings of the 2006 international symposium on Software testing and analysis
Predicting fault-prone components in a java legacy system

Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
Predicting component failures at design time

Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
Automating algorithms for the identification of fault-prone files

Proceedings of the 2007 international symposium on Software testing and analysis

Software engineering research: from cradle to grave

Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
Misclassification cost-sensitive fault prediction models

PROMISE '09 Proceedings of the 5th International Conference on Predictor Models in Software Engineering
A systematic and comprehensive investigation of methods to build and evaluate fault prediction models

Journal of Systems and Software
Comparing the effectiveness of several modeling methods for fault prediction

Empirical Software Engineering
Predicting vulnerable software components with dependency graphs

Proceedings of the 6th International Workshop on Security Measurements and Metrics
Using complexity, coupling, and cohesion metrics as early indicators of vulnerabilities

Journal of Systems Architecture: the EUROMICRO Journal
A genetic algorithm to configure support vector machines for predicting fault-prone components

PROFES'11 Proceedings of the 12th international conference on Product-focused software process improvement
A framework for defect prediction in specific software project contexts

CEE-SET'08 Proceedings of the Third IFIP TC 2 Central and East European conference on Software engineering techniques
Searching for rules to detect defective modules: A subgroup discovery approach

Information Sciences: an International Journal
A further analysis on the use of Genetic Algorithm to configure Support Vector Machines for inter-release fault prediction

Proceedings of the 27th Annual ACM Symposium on Applied Computing
Comparing the performance of fault prediction models which report multiple performance measures: recomputing the confusion matrix

Proceedings of the 8th International Conference on Predictive Models in Software Engineering
Measuring architecture quality by structure plus history analysis

Proceedings of the 2013 International Conference on Software Engineering
DConfusion: a technique to allow cross study performance evaluation of fault prediction studies

Automated Software Engineering

Quantified Score

Hi-index	0.00

Visualization

Abstract

Many fault prediction models have been proposed in the software engineering literature, and their success evaluated according to various metrics that are widely used in the statistics community. To be able to make meaningful comparisons among the proposed models, it is important that the metrics assess meaningful properties of the predictions. We examine several of the more common metrics, discuss the advantages and disadvantages of each, and illustrate their application to predictions made on a large industrial system. We conclude that the most useful metrics are the percentage of faults that occur in the predicted most fault-prone files, and the Type II misclassification rate.