An empirical validation of software cost estimation models
Communications of the ACM
Robust regression for developing software estimation models
Journal of Systems and Software
Effort estimation using analogy
Proceedings of the 18th international conference on Software engineering
Technical note: some properties of splitting criteria
Machine Learning
Inter-item correlations among function points
ICSE '93 Proceedings of the 15th international conference on Software Engineering
Estimating Software Project Effort Using Analogies
IEEE Transactions on Software Engineering
Selection of the optimal prototype subset for 1-NN classification
Pattern Recognition Letters
Comparing Software Prediction Techniques Using Simulation
IEEE Transactions on Software Engineering - Special section on the seventh international software metrics symposium
Software Engineering Economics
Software Engineering Economics
An Empirical Study of Analogy-based Software Effort Estimation
Empirical Software Engineering
A Comparative Study of Cost Estimation Models for Web Hypermedia Applications
Empirical Software Engineering
Benchmarking Attribute Selection Techniques for Discrete Class Data Mining
IEEE Transactions on Knowledge and Data Engineering
A Simulation Study of the Model Evaluation Criterion MMRE
IEEE Transactions on Software Engineering
Distribution Patterns of Effort Estimations
EUROMICRO '04 Proceedings of the 30th EUROMICRO Conference
Introduction to Machine Learning (Adaptive Computation and Machine Learning)
Introduction to Machine Learning (Adaptive Computation and Machine Learning)
Reliability and Validity in Comparative Studies of Software Prediction Models
IEEE Transactions on Software Engineering
Optimal Project Feature Weights in Analogy-Based Cost Estimation: Improvement and Limitations
IEEE Transactions on Software Engineering
Discretization from data streams: applications to histograms and data mining
Proceedings of the 2006 ACM symposium on Applied computing
A comparative study of attribute weighting heuristics for effort estimation by analogy
Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
Cross versus Within-Company Cost Estimation Studies: A Systematic Review
IEEE Transactions on Software Engineering
Decision Support Analysis for Software Effort Estimation by Analogy
PROMISE '07 Proceedings of the Third International Workshop on Predictor Models in Software Engineering
Finding Prototypes For Nearest Neighbor Classifiers
IEEE Transactions on Computers
IEEE Transactions on Software Engineering
Experiments with Analogy-X for Software Cost Estimation
ASWEC '08 Proceedings of the 19th Australian Conference on Software Engineering
Empirical evaluation of analogy-x for software cost estimation
Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement
Analogy-X: Providing Statistical Inference to Analogy-Based Software Cost Estimation
IEEE Transactions on Software Engineering
A study of project selection and feature weighting for analogy based software cost estimation
Journal of Systems and Software
Stable rankings for different effort models
Automated Software Engineering
Case-based reasoning vs parametric models for software quality optimization
Proceedings of the 6th International Conference on Predictive Models in Software Engineering
A review of studies on expert estimation of software development effort
Journal of Systems and Software
Hi-index | 0.00 |
Background: Conclusion Instability in software effort estimation (SEE) refers to the inconsistent results produced by a diversity of predictors using different datasets. This is largely due to the "ranking instability" problem, which is highly related to the evaluation criteria and the subset of the data being used. Aim: To determine stable rankings of different predictors. Method: 90 predictors are used with 20 datasets and evaluated using 7 performance measures, whose results are subject to Wilcoxon rank test (95 %). These results are called the "aggregate results". The aggregate results are challenged by a sanity check, which focuses on a single error measure (MRE) and uses a newly developed evaluation algorithm called CLUSTER. These results are called the "specific results." Results: Aggregate results show that: (1) It is now possible to draw stable conclusions about the relative performance of SEE predictors; (2) Regression trees or analogy-based methods are the best performers. The aggregate results are also confirmed by the specific results of the sanity check. Conclusion: This study offers means to address the conclusion instability issue in SEE, which is an important finding for empirical software engineering.