On the improvement of a fault classification scheme with implications for white-box testing

  • Authors:

  • Shimul Kumar Nath;Robert Merkel;Man Fai Lau

  • Affiliations:

  • Swinburne University of Technology, Hawthorn, Australia;Monash University, Clayton, Australia;Swinburne University of Technology, Hawthorn, Australia

  • Venue:
  • Proceedings of the 27th Annual ACM Symposium on Applied Computing
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

Different testing techniques can be more or less effective on different fault types; therefore, testing methods that are most likely to detect the most common fault types should be preferred. However, to enable such a selection of testing methods, a suitable and effective fault classification scheme is essential. Much software testing research proposes techniques to generate test cases and evaluates these techniques based on some hypothesized fault classification scheme. However, there is a lack in the justification of whether such 'hypothesized faults' are realistic and how often these faults occur. Recently, Pan et al. analyzed the syntactic changes in the source code, made in fixing faults, of 7 open source software projects implemented in Java. Based on their experience, they proposed a fault classification scheme with relative frequencies of each fault type. As always, readers may question whether the resulting fault classification is reasonable and appropriate. Hence, we applied their method to Checkstyle, another open source Java program, as the subject of our study, hoping to validate the appropriateness of their fault classification scheme, with particular application for selecting testing methods. While we generally found their classification scheme reasonable, we also noted that some faults could be classified in multiple ways. We also found that the frequencies of fault categories in Checkstyle were significantly different to the seven systems studied by Pan et al., which had all shown to have quite similar frequencies. We identified several potential improvements to their classification, permitting the classification of a larger proportion of faults. We have identified new implications of the combined results for white-box testing, and proposed follow-up studies to examine these issues in more detail.