Better sure than safe? Over-confidence in judgement based software development effort prediction intervals

  • Authors:

  • Magne JøRgensen;Karl Halvor Teigen;Kjetil MoløKken

  • Affiliations:

  • Simula Research Laboratory, Software Engineering, P.O. Box 134, 1325 Lysaker, Norway;Department of Psychology, University of Oslo, P.O. Box 1094, Blindern, 0317 Oslo, Norway;Simula Research Laboratory, Software Engineering, P.O. Box 134, 1325 Lysaker, Norway

  • Venue:
  • Journal of Systems and Software
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

The uncertainty of a software development effort estimate can be indicated through a prediction interval (PI), i.e., the estimated minimum and maximum effort corresponding to a specific confidence level. For example, a project manager may be ''90% confident'' or believe that is it ''very likely'' that the effort required to complete a project will be between 8000 and 12,000 work-hours. This paper describes results from four studies (Studies A-D) on human judgement (expert) based PIs of software development effort. Study A examines the accuracy of the PIs in real software projects. The results suggest that the PIs were generally much too narrow to reflect the chosen level of confidence, i.e., that there was a strong over-confidence. Studies B-D try to understand the reasons for the observed over-confidence. Study B examines the possibility that the over-confidence is related to type of experience or estimation process. Study C examines the possibility that the concept of confidence level is difficult to interpret for software estimators. Finally, Study D examines the possibility that there are unfortunate feedback mechanisms that reward over-confidence.