Software engineering metrics and models
Software engineering metrics and models
Function Points Analysis: An Empirical Study of Its Measurement Processes
IEEE Transactions on Software Engineering
Estimating Software Project Effort Using Analogies
IEEE Transactions on Software Engineering
A Procedure for Analyzing Unbalanced Datasets
IEEE Transactions on Software Engineering
A Controlled Experiment to Assess the Benefits of Estimating with Analogy and Regression Models
IEEE Transactions on Software Engineering
A replicated assessment and comparison of common software cost modeling techniques
Proceedings of the 22nd international conference on Software engineering
An investigation of machine learning based prediction systems
Journal of Systems and Software - Special issue on empirical studies of software development and evolution
A Simulation Tool for Efficient Analogy Based Cost Estimation
Empirical Software Engineering
Using Public Domain Metrics To Estimate Software Development Effort
METRICS '01 Proceedings of the 7th International Symposium on Software Metrics
Reliability and Validity in Comparative Studies of Software Prediction Models
IEEE Transactions on Software Engineering
IEEE Transactions on Software Engineering
Modern Applied Statistics with S
Modern Applied Statistics with S
Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement
Why comparative effort prediction studies may be invalid
PROMISE '09 Proceedings of the 5th International Conference on Predictor Models in Software Engineering
Visual comparison of software cost estimation models by regression error characteristic analysis
Journal of Systems and Software
Empirical Software Engineering
Modeling the relationship between software effort and size using deming regression
Proceedings of the 6th International Conference on Predictive Models in Software Engineering
Empirical Software Engineering
StatREC: a graphical user interface tool for visual hypothesis testing of cost prediction models
Proceedings of the 8th International Conference on Predictive Models in Software Engineering
Alternative methods using similarities in software effort estimation
Proceedings of the 8th International Conference on Predictive Models in Software Engineering
Grey relational effort analysis technique using robust regression methods for individual projects
International Journal of Computational Intelligence Studies
Hi-index | 0.00 |
The accurate software cost prediction is a research topic that has attracted much of the interest of the software engineering community during the latest decades. A large part of the research efforts involves the development of statistical models based on historical data. Since there are a lot of models that can be fitted to certain data, a crucial issue is the selection of the most efficient prediction model. Most often this selection is based on comparisons of various accuracy measures that are functions of the model's relative errors. However, the usual practice is to consider as the most accurate prediction model the one providing the best accuracy measure without testing if this superiority is in fact statistically significant. This policy can lead to unstable and erroneous conclusions since a small change in the data is able to turn over the best model selection. On the other hand, the accuracy measures used in practice are statistics with unknown probability distributions, making the testing of any hypothesis, by the traditional parametric methods, problematic. In this paper, the use of statistical simulation tools is proposed in order to test the significance of the difference between the accuracy of two prediction methods: regression and estimation by analogy. The statistical simulation procedures involve permutation tests and bootstrap techniques for the construction of confidence intervals for the difference of measures. Four known datasets are used for experimentation in order to validate the results and make comparisons between the simulation methods and the traditional parametric and non-parametric procedures.