Concepts in configuration management systems
SCM '91 Proceedings of the 3rd international workshop on Software configuration management
The anatomy of a large-scale hypertextual Web search engine
WWW7 Proceedings of the seventh international conference on World Wide Web 7
A Critique of Software Defect Prediction Models
IEEE Transactions on Software Engineering
Validation, Verification, and Testing of Computer Software
ACM Computing Surveys (CSUR)
Elements of Software Science (Operating and programming systems series)
Elements of Software Science (Operating and programming systems series)
Proceedings of the 24th International Conference on Software Engineering
A Metrics Suite for Object Oriented Design
IEEE Transactions on Software Engineering
Predicting the Location and Number of Faults in Large Software Systems
IEEE Transactions on Software Engineering
Building Defect Prediction Models in Practice
IEEE Software
Predicting fault-prone components in a java legacy system
Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
Using Historical In-Process and Product Metrics for Early Estimation of Software Failures
ISSRE '06 Proceedings of the 17th International Symposium on Software Reliability Engineering
Data Mining Static Code Attributes to Learn Defect Predictors
IEEE Transactions on Software Engineering
IEEE Transactions on Software Engineering
The influence of organizational structure on software quality: an empirical case study
Proceedings of the 30th international conference on Software engineering
Implications of ceiling effects in defect predictors
Proceedings of the 4th international workshop on Predictor models in software engineering
Nearest neighbor sampling for cross company defect predictors: abstract only
DEFECTS '08 Proceedings of the 2008 workshop on Defects in large software systems
Ensemble of software defect predictors: a case study
Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement
On the effectiveness of early life cycle defect prediction with Bayesian Nets
Empirical Software Engineering
IEEE Transactions on Software Engineering
Analysis of Naive Bayes' assumptions on software fault data: An empirical study
Data & Knowledge Engineering
Software Defect Prediction Using Call Graph Based Ranking (CGBR) Framework
SEAA '08 Proceedings of the 2008 34th Euromicro Conference Software Engineering and Advanced Applications
Optimized Resource Allocation for Software Release Planning
IEEE Transactions on Software Engineering
PROMISE '09 Proceedings of the 5th International Conference on Predictor Models in Software Engineering
Cross-project defect prediction: a large scale experiment on data vs. domain vs. process
Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
On the relative value of cross-company and within-company data for defect prediction
Empirical Software Engineering
An empirical study of the effects of personality in pair programming using the five-factor model
ESEM '09 Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement
Reducing false alarms in software defect prediction by decision threshold optimization
ESEM '09 Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement
Dione: an integrated measurement and defect prediction solution
Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering
Empirical evaluation of the effects of mixed project data on learning defect predictors
Information and Software Technology
Hi-index | 0.00 |
Context: Building defect prediction models in large organizations has many challenges due to limited resources and tight schedules in the software development lifecycle. It is not easy to collect data, utilize any type of algorithm and build a permanent model at once. We have conducted a study in a large telecommunications company in Turkey to employ a software measurement program and to predict pre-release defects. Based on our prior publication, we have shared our experience in terms of the project steps (i.e. challenges and opportunities). We have further introduced new techniques that improve our earlier results. Objective: In our previous work, we have built similar predictors using data representative for US software development. Our task here was to check if those predictors were specific solely to US organizations or to a broader class of software. Method: We have presented our approach and results in the form of an experience report. Specifically, we have made use of different techniques for improving the information content of the software data and the performance of a Naive Bayes classifier in the prediction model that is locally tuned for the company. We have increased the information content of the software data by using module dependency data and improved the performance by adjusting the hyper-parameter (decision threshold) of the Naive Bayes classifier. We have reported and discussed our results in terms of defect detection rates and false alarms. We also carried out a cost-benefit analysis to show that our approach can be efficiently put into practice. Results: Our general result is that general defect predictors, which exist across a wide range of software (in both US and Turkish organizations), are present. Our specific results indicate that concerning the organization subject to this study, the use of version history information along with code metrics decreased false alarms by 22%, the use of dependencies between modules further reduced false alarms by 8%, and the decision threshold optimization for the Naive Bayes classifier using code metrics and version history information further improved false alarms by 30% in comparison to a prediction using only code metrics and a default decision threshold. Conclusion: Implementing statistical techniques and machine learning on a real life scenario is a difficult yet possible task. Using simple statistical and algorithmic techniques produces an average detection rate of 88%. Although using dependency data improves our results, it is difficult to collect and analyze such data in general. Therefore, we would recommend optimizing the hyper-parameter of the proposed technique, Naive Bayes, to calibrate the defect prediction model rather than employing more complex classifiers. We also recommend that researchers who explore statistical and algorithmic methods for defect prediction should spend less time on their algorithms and more time on studying the pragmatic considerations of large organizations.