An approach for experimentally evaluating effectiveness and efficiency of coverage criteria for software testing

  • Authors:

  • Atul Gupta;Pankaj Jalote

  • Affiliations:

  • Indian Institute of Technology, 208016, Kanpur, India;Indian Institute of Technology, 208016, Kanpur, India and Indian Institute of Technology Delhi, 110016, Delhi, India

  • Venue:
  • International Journal on Software Tools for Technology Transfer (STTT)
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Experimental work in software testing has generally focused on evaluating the effectiveness and effort requirements of various coverage criteria. The important issue of testing efficiency has not been sufficiently addressed. In this paper, we describe an approach for comparing the effectiveness and efficiency of test coverage criteria using mutation analysis. For each coverage criterion under study, we generate multiple coverage-adequate minimal test suites for a test-program from a test-pool, which are then executed on a set of systematically generated program mutants to obtain the fault data. We demonstrate the applicability of the proposed approach by describing the results of an experiment comparing the three code-based testing criteria, namely, block coverage, branch coverage, and predicate coverage. Our results suggest that there is a trade-off between effectiveness and efficiency of a coverage criterion. Specifically, the predicate coverage criterion was found to be most effective but least efficient whereas the block coverage criterion was most efficient but least effective. We observed high variability in the performance of block test suites whereas branch and predicate test suites were relatively consistent. Overall results suggest that the branch coverage criterion performs consistently with good efficiency and effectiveness, and it appears to be the most viable option for code-based control flow testing.