An Empirical Comparison of Two Safe Regression Test Selection Techniques

• Authors: Phyllis G. Frankl;Gregg Rothermel;Kent Sayre;Filippos I. Vokolos
• Venue: ISESE '03 Proceedings of the 2003 International Symposium on Empirical Software Engineering
• Year: 2003

Regression test selection techniques reduce the costof regression testing by selecting a subset of an existingtest suite to use in retesting a modified program.Safe regression test selection techniques guarantee (underspecific conditions) that the selected subset will notomit faults that could have been revealed by the entiresuite. Many regression test selection techniques havebeen described in the literature. Empirical studies ofsome of these techniques have shown that they can bebeneficial, but only a few studies have empirically compareddifferent techniques, and fewer still have consideredsafe techniques. In this paper, we report the resultsof a comparative empirical study of implementations oftwo safe regression test selection techniques: DejaVuand Pythia. Our results show that, despite differencesin their approaches, and despite the theoreticallygreater ability of DejaVu to select smaller test suitesthan Pythia, the two techniques often selected equivalenttest suites in practice, at comparable costs. Theseresults suggest that factors such as ease of implementation,generality, and availability of supporting tools anddata may play a greater role than cost-effectiveness forpractitioners choosing between these techniques.