A practical approach to feature selection
ML92 Proceedings of the ninth international workshop on Machine learning
Estimating attributes: analysis and extensions of RELIEF
ECML-94 Proceedings of the European conference on machine learning on Machine Learning
A Validation of Object-Oriented Design Metrics as Quality Indicators
IEEE Transactions on Software Engineering
Predicting Fault-Prone Software Modules in Telephone Switches
IEEE Transactions on Software Engineering
Wrappers for feature subset selection
Artificial Intelligence - Special issue on relevance
A Unified Framework for Coupling Measurement in Object-Oriented Systems
IEEE Transactions on Software Engineering
Proceedings of the 20th international conference on Software engineering
Predicting Fault Incidence Using Software Change History
IEEE Transactions on Software Engineering
The prediction of faulty classes using object-oriented design metrics
Journal of Systems and Software
The distribution of faults in a large industrial software system
ISSTA '02 Proceedings of the 2002 ACM SIGSOFT international symposium on Software testing and analysis
Simple Explanation of the No Free Lunch Theorem of Optimization
Cybernetics and Systems Analysis
A Comparative Study of Ordering and Classification of Fault-ProneSoftware Modules
Empirical Software Engineering
A Metrics Suite for Object Oriented Design
IEEE Transactions on Software Engineering
Quantitative Analysis of Faults and Failures in a Complex Software System
IEEE Transactions on Software Engineering
Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning
ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
IEEE Transactions on Software Engineering
Using benchmarking to advance research: a challenge to software engineering
Proceedings of the 25th International Conference on Software Engineering
Detection of software modules with high debug code churn in a very large legacy system
ISSRE '96 Proceedings of the The Seventh International Symposium on Software Reliability Engineering
A Study on the Current State of the Art in Tool-Supported UML-Based Static Reverse Engineering
WCRE '02 Proceedings of the Ninth Working Conference on Reverse Engineering (WCRE'02)
Populating a Release History Database from Version Control and Bug Tracking Systems
ICSM '03 Proceedings of the International Conference on Software Maintenance
Developing Fault Predictors for Evolving Software Systems
METRICS '03 Proceedings of the 9th International Symposium on Software Metrics
Benchmarking Attribute Selection Techniques for Discrete Class Data Mining
IEEE Transactions on Knowledge and Data Engineering
A Simulation Study of the Model Evaluation Criterion MMRE
IEEE Transactions on Software Engineering
ISSTA '04 Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis
Use of relative code churn measures to predict system defect density
Proceedings of the 27th international conference on Software engineering
Static analysis tools as early indicators of pre-release defect density
Proceedings of the 27th international conference on Software engineering
Predicting the Location and Number of Faults in Large Software Systems
IEEE Transactions on Software Engineering
Reliability and Validity in Comparative Studies of Software Prediction Models
IEEE Transactions on Software Engineering
Moose: an agile reengineering environment
Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering
The Top Ten List: Dynamic Fault Prediction
ICSM '05 Proceedings of the 21st IEEE International Conference on Software Maintenance
Empirical Validation of Object-Oriented Metrics on Open Source Software for Fault Prediction
IEEE Transactions on Software Engineering
Mining metrics to predict component failures
Proceedings of the 28th international conference on Software engineering
Predicting fault-prone components in a java legacy system
Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
Statistical Comparisons of Classifiers over Multiple Data Sets
The Journal of Machine Learning Research
Data Mining Static Code Attributes to Learn Defect Predictors
IEEE Transactions on Software Engineering
Predicting Faults from Cached History
ICSE '07 Proceedings of the 29th international conference on Software Engineering
Predicting Defects for Eclipse
PROMISE '07 Proceedings of the Third International Workshop on Predictor Models in Software Engineering
Modeling the Effect of Size on Defect Proneness for Open-Source Software
PROMISE '07 Proceedings of the Third International Workshop on Predictor Models in Software Engineering
Automating algorithms for the identification of fault-prone files
Proceedings of the 2007 international symposium on Software testing and analysis
Improving defect prediction using temporal features and non linear models
Ninth international workshop on Principles of software evolution: in conjunction with the 6th ESEC/FSE joint meeting
Predicting vulnerable software components
Proceedings of the 14th ACM conference on Computer and communications security
Proceedings of the 30th international conference on Software engineering
Predicting defects using network analysis on dependency graphs
Proceedings of the 30th international conference on Software engineering
Using the Conceptual Cohesion of Classes for Fault Prediction in Object-Oriented Systems
IEEE Transactions on Software Engineering
Theory of relative defect proneness
Empirical Software Engineering
Techniques for evaluating fault prediction models
Empirical Software Engineering
IEEE Transactions on Software Engineering
Can developer-module networks predict failures?
Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering
Is it a bug or an enhancement?: a text-based approach to classify change requests
CASCON '08 Proceedings of the 2008 conference of the center for advanced studies on collaborative research: meeting of minds
Revisiting the evaluation of defect prediction models
PROMISE '09 Proceedings of the 5th International Conference on Predictor Models in Software Engineering
Predicting build failures using social network analysis on developer communication
ICSE '09 Proceedings of the 31st International Conference on Software Engineering
Predicting faults using the complexity of code changes
ICSE '09 Proceedings of the 31st International Conference on Software Engineering
Does calling structure information improve the accuracy of fault prediction?
MSR '09 Proceedings of the 2009 6th IEEE International Working Conference on Mining Software Repositories
Cross-project defect prediction: a large scale experiment on data vs. domain vs. process
Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
Fair and balanced?: bias in bug-fix datasets
Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
On the relative value of cross-company and within-company data for defect prediction
Empirical Software Engineering
Journal of Systems and Software
Using differences among replications of software engineering experiments to gain knowledge
ESEM '09 Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement
Distributed and Collaborative Software Evolution Analysis with Churrasco
Science of Computer Programming
Defect prediction from static code features: current results, limitations, new approaches
Automated Software Engineering
Replication of defect prediction studies: problems, pitfalls and recommendations
Proceedings of the 6th International Conference on Predictive Models in Software Engineering
Revisiting common bug prediction findings using effort-aware models
ICSM '10 Proceedings of the 2010 IEEE International Conference on Software Maintenance
Physical and conceptual identifier dispersion: Measures and relation to fault proneness
ICSM '10 Proceedings of the 2010 IEEE International Conference on Software Maintenance
Effort-Aware Defect Prediction Models
CSMR '10 Proceedings of the 2010 14th European Conference on Software Maintenance and Reengineering
Are popular classes more defect prone?
FASE'10 Proceedings of the 13th international conference on Fundamental Approaches to Software Engineering
Regularities in learning defect predictors
PROFES'10 Proceedings of the 11th international conference on Product-Focused Software Process Improvement
Content classification of development emails
Proceedings of the 34th International Conference on Software Engineering
A learning-to-rank algorithm for constructing defect prediction models
IDEAL'12 Proceedings of the 13th international conference on Intelligent Data Engineering and Automated Learning
Proceedings of the ACM-IEEE international symposium on Empirical software engineering and measurement
A study of subgroup discovery approaches for defect prediction
Information and Software Technology
Studying software evolution using topic models
Science of Computer Programming
Bug prediction using entropy-based measures
International Journal of Knowledge Engineering and Data Mining
Software defect prediction using relational association rule mining
Information Sciences: an International Journal
Hi-index | 0.00 |
Reliably predicting software defects is one of the holy grails of software engineering. Researchers have devised and implemented a plethora of defect/bug prediction approaches varying in terms of accuracy, complexity and the input data they require. However, the absence of an established benchmark makes it hard, if not impossible, to compare approaches. We present a benchmark for defect prediction, in the form of a publicly available dataset consisting of several software systems, and provide an extensive comparison of well-known bug prediction approaches, together with novel approaches we devised. We evaluate the performance of the approaches using different performance indicators: classification of entities as defect-prone or not, ranking of the entities, with and without taking into account the effort to review an entity. We performed three sets of experiments aimed at (1) comparing the approaches across different systems, (2) testing whether the differences in performance are statistically significant, and (3) investigating the stability of approaches across different learners. Our results indicate that, while some approaches perform better than others in a statistically significant manner, external validity in defect prediction is still an open problem, as generalizing results to different contexts/learners proved to be a partially unsuccessful endeavor.