An Empirical Study on Testing and Fault Tolerance for Software Reliability Engineering

  • Authors:
  • Michael R. Lyu;Zubin Huang;Sam K. S. Sze;Xia Cai

  • Affiliations:
  • -;-;-;-

  • Venue:
  • ISSRE '03 Proceedings of the 14th International Symposium on Software Reliability Engineering
  • Year:
  • 2003

Quantified Score

Hi-index 0.00

Visualization

Abstract

Software testing and software fault tolerance are twomajor techniques for developing reliable softwaresystems, yet limited empirical data are available in theliterature to evaluate their effectiveness. We conducted amajor experiment to engage 34 programming teams toindependently develop multiple software versions for anindustry-scale critical flight application, and collectedfaults detected in these program versions. To evaluatethe effectiveness of software testing and software faulttolerance, mutants were created by injecting real faultsoccurred in the development stage. The nature,manifestation, detection, and correlation of these faultswere carefully investigated. The results show thatcoverage testing is generally an effective mean todetecting software faults, but the effectiveness of testingcoverage is not equivalent to that of mutation coverage,which is a more truthful indicator of testing quality. Wealso found that exact faults found among versions arevery limited. This result supports software fault toleranceby design diversity as a creditable approach for softwarereliability engineering. Finally we conducted domainanalysis approach for test case generation, andconcluded that it is a promising technique for softwaretesting purpose.