The cost of errors in software development: evidence from industry

  • Authors:

  • J. Christopher Westland

  • Affiliations:

  • Department of Information and Systems Management, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong

  • Venue:
  • Journal of Systems and Software
  • Year:
  • 2002

Quantified Score

Hi-index 0.00

Visualization

Abstract

The search for and correction of errors in software are often time consuming and expensive components of the total cost of software development. The current research investigates to what extent these costs of software error detection and correction contribute to the total cost of software. We initiated the research reported here with the collection of a sample of transactions recording progress on one phase of development of a set of software programs. Each of these projects represented the completion of an identical phase of development (i.e., country localisation) for a different country. This enabled each project to be compared with the other, and provided an unusually high degree of control over the data collection and analysis in real-world empirical study. The research findings relied on programmers' self-assessment of the severity of errors discovered. It found that serious errors have less influence on total cost than errors that were classified as less serious but which occurred with more frequency once these less serious errors are actually resolved and corrected. The research suggests one explanation - that programmers have greater discretion in how and when to resolve these less severe errors. The data supports the hypothesis that errors generate significant software development costs if their resolution requires system redesign. Within the context of the research, it was concluded that uncorrected errors become exponentially more costly with each phase in which they are unresolved, which is consistent with earlier findings in the literature. The research also found that the number of days that a project is open is a log-linear predictor of the number of software errors that will be discovered, implying a bias in error discovery over time. This implies that testing results need to be interpreted in light of the length of testing, and that in practice, tests should take place both before and after systems release.