Combination of software development effort prediction intervals: why, when and how?

  • Authors:

  • Magne Jørgensen;Kjetil Moløkken

  • Affiliations:

  • Simula Research Laboratory, Lysaker, Norway;University of Oslo, Oslo, Norway

  • Venue:
  • SEKE '02 Proceedings of the 14th international conference on Software engineering and knowledge engineering
  • Year:
  • 2002

Quantified Score

Hi-index 0.00

Visualization

Abstract

The uncertainty of a software development effort estimate may be described through a prediction interval, e.g., that the most likely use of effort is 1.500 work-hours and that it is 90 % probable (90% confidence level) that the actual use of effort will be between 1.000 (minimum) and 2.000 (maximum) work-hours. Previous studies suggest that software development effort prediction intervals are, on average, much too narrow to reflect high confidence levels, i.e., the uncertainty is under-estimated. This paper analyses when and how a combination of several individual prediction intervals of the same task improves the correspondence between hit rate and confidence level of effort prediction intervals. We analyse three combination strategies: (1) Average of the individual minimum and maximum values, (2) Maximum and minimum of the individual maximum and minimum values, and (3) Group process (discussion) based prediction intervals. Based on an empirical study with software professionals we found that strategy (1) did not lead to much correspondence improvement compared with the individual prediction intervals, mainly because of a, as expected, strong individual bias towards too narrow prediction intervals. Strategy (2) and (3) both improved the correspondence. However, Strategy (3) used the uncertainty information more efficiently, i.e., had narrower prediction intervals for the same hit rate. Our empirical results suggest that group discussion based combination of prediction intervals should be used instead of "mechanical" combinations of individual prediction intervals. Clearly, there is no best combination strategy for all prediction interval situations, and the choice of strategy should be based on an investigation of factors that impact the usefulness of a strategy.