Modeling the Effects of Combining Diverse Software Fault Detection Techniques

  • Authors:

  • Bev Littlewood;Peter T. Popov;Lorenzo Strigini;Nick Shryane

  • Affiliations:

  • City Univ., London, UK;City Univ., London, UK;City Univ., London, UK;Univ. of Hull, Hull, UK

  • Venue:
  • IEEE Transactions on Software Engineering
  • Year:
  • 2000

Quantified Score

Hi-index 0.01

Visualization

Abstract

The software engineering literature contains many studies of the efficacy of fault finding techniques. Few of these, however, consider what happens when several different techniques are used together. We show that the effectiveness of such multitechnique approaches depends upon quite subtle interplay between their individual efficacies and dependence between them. The modeling tool we use to study this problem is closely related to earlier work on software design diversity. The earliest of these results showed that, under quite plausible assumptions, it would be unreasonable even to expect software versions that were developed 驴truly independently驴 to fail independently of one another. The key idea here was a 驴difficulty function驴 over the input space. Later work extended these ideas to introduce a notion of 驴forced驴 diversity, in which it became possible to obtain system failure behavior better even than could be expected if the versions failed independently. In this paper, we show that many of these results for design diversity have counterparts in diverse fault detection in a single software version. We define measures of fault finding effectiveness and of diversity and show how these might be used to give guidance for the optimal application of different fault finding procedures to a particular program. We show that the effects upon reliability of repeated applications of a particular fault finding procedure are not statistically independent驴in fact, such an incorrect assumption of independence will always give results that are too optimistic. For diverse fault finding procedures, on the other hand, things are different: Here it is possible for effectiveness to be even greater than it would be under an assumption of statistical independence. We show that diversity of fault finding procedures is, in a precisely defined way, 驴a good thing驴 and should be applied as widely as possible. The new model and its results are illustrated using some data from an experimental investigation into diverse fault finding on a railway signalling application.