Revisiting the evaluation of defect prediction models
PROMISE '09 Proceedings of the 5th International Conference on Predictor Models in Software Engineering
Misclassification cost-sensitive fault prediction models
PROMISE '09 Proceedings of the 5th International Conference on Predictor Models in Software Engineering
Cost curve analysis of biometric system performance
BTAS'09 Proceedings of the 3rd IEEE international conference on Biometrics: Theory, applications and systems
Variance analysis in software fault prediction models
ISSRE'09 Proceedings of the 20th IEEE international conference on software reliability engineering
Defect prediction using social network analysis on issue repositories
Proceedings of the 2011 International Conference on Software and Systems Process
On the dataset shift problem in software engineering prediction models
Empirical Software Engineering
| Hi-index | 0.00 |
Prediction of fault prone software components is one of the most researched problems in software engineering. Many statistical techniques have been proposed but there is no consensus on the methodology to select the "best model" for the specific project. In this paper, we introduce and discuss the merits of cost curve analysis of fault prediction models. Cost curves allow software quality engineers to introduce project-specific cost of module misclassification into model evaluation. Classifying a software module as fault-prone implies the application of some verification activities, thus adding to the development cost. Misclassifying a module as fault free carries the risk of system failure, also associated with cost implications. Through the analysis of sixteen projects from public repositories, we observe that software quality does not necessarily benefit from the prediction of fault prone components. The inclusion of misclassification cost in model evaluation may indicate that even the "best" models achieve performance no better than trivial classification. Our results support a recommendation to adopt cost curves as one of the standard methods for software quality model performance evaluation.