Automated oracle creation support, or: how I learned to stop worrying about fault propagation and love mutation testing

  • Authors:
  • Matt Staats;Gregory Gay;Mats P. E. Heimdahl

  • Affiliations:
  • KAIST, South Korea;University of Minnesota, USA;University of Minnesota, USA

  • Venue:
  • Proceedings of the 34th International Conference on Software Engineering
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

In testing, the test oracle is the artifact that determines whether an application under test executes correctly. The choice of test oracle can significantly impact the effectiveness of the testing process. However, despite the prevalence of tools that support the selection of test inputs, little work exists for supporting oracle creation. In this work, we propose a method of supporting test oracle creation. This method automatically selects the oracle data â聙聰 the set of variables monitored during testingâ聙聰for expected value test oracles. This approach is based on the use of mutation analysis to rank variables in terms of fault-finding effectiveness, thus automating the selection of the oracle data. Experiments over four industrial examples demonstrate that our method may be a cost-effective approach for producing small, effective oracle data, with fault finding improvements over current industrial best practice of up to 145.8% observed.