An evaluation of change-based coverage criteria

  • Authors:

  • Marc Fisher, II;Jan Wloka;Frank Tip;Barbara G. Ryder;Alexander Luchansky

  • Affiliations:

  • Virginia Tech, Blacksburg, VA, USA;IBM Rational Research Lab, Zurich, Switzerland;IBM Research, Yorktown Heights, NY, USA;Virginia Tech, Blacksburg, VA, USA;Vanguard Group, Inc., Malvern, PA, USA

  • Venue:
  • Proceedings of the 10th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Various coverage criteria are commonly used to assess the quality of test suites, but achieving full coverage according to these criteria is often impossible or impractical. Our research starts from the popular assumption that a disproportionate number of faults is likely to reside in recently changed code. Based on this assumption, we propose several change-based coverage criteria that reflect to what extent changes with respect to a previous program version are exercised by a test suite. In a set of experiments on programs from the SIR repository, we found change-based criteria to reveal faults bet- ter than traditional criteria, and to enable the construction of much smaller test suites with similar fault detection effectiveness. We also report on a case study that shows that achieving 100% coverage according to a change-based criterion is feasible and that by doing so we were able to find additional faults, including one fault that was not intentionally seeded in the subject program.