A Comparison of Coverage-Based and Distribution-Based Techniques for Filtering and Prioritizing Test Cases

  • Authors:
  • David Leon;Andy Podgurski

  • Affiliations:
  • -;-

  • Venue:
  • ISSRE '03 Proceedings of the 14th International Symposium on Software Reliability Engineering
  • Year:
  • 2003

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper presents an empirical comparison of fourdifferent techniques for filtering large test suites: testsuite minimization, prioritization by additional coverage,cluster filtering with one-per-cluster sampling, andfailure pursuit sampling. The first two techniques arebased on selecting subsets that maximize code coverageas quickly as possible, while the latter two are based onanalyzing the distribution of the tests' execution profiles.These techniques were compared with data sets obtainedfrom three large subject programs: the GCC, Jikes, andjavac compilers. The results indicate that distribution-basedtechniques can be as efficient or more efficient forrevealing defects than coverage-based techniques, butthat the two kinds of techniques are also complementaryin the sense that they find different defects. Accordingly,some simple combinations of these techniques wereevaluated for use in test case prioritization. The resultsindicate that these techniques can create more efficientprioritizations than those generated using prioritizationby additional coverage.