Differential testing: a new approach to change detection

  • Authors:
  • Robert B. Evans;Alberto Savoia

  • Affiliations:
  • Agitar Laboratories, Mountain View, CA;Agitar Software, Mountain View, CA

  • Venue:
  • Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

Regression testing, as it's commonly practiced, is unsound due to inconsistent test repair and test addition. This paper presents a new technique, differential testing, that alleviates the test repair problem and detects more changes than regression testing alone. Differential testing works by creating test suites for both the original system and the modified system and contrasting both versions of the system with these two suites. Differential testing is made possible by recent advances in automated unit test generation. Furthermore, it makes automated test generators more useful because it abstracts away the interpretation and management of large volumes of tests by focusing on the changes between test suites. In our preliminary empirical study of 3 subjects, differential testing discovered 21%, 34%, and 21% more behavior changes than regression testing alone.