On test repair using symbolic execution

  • Authors:

  • Brett Daniel;Tihomir Gvero;Darko Marinov

  • Affiliations:

  • University of Illinois at Urbana-Champaign, Urbana, IL, USA;Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland;University of Illinois at Urbana-Champaign, Urbana, IL, USA

  • Venue:
  • Proceedings of the 19th international symposium on Software testing and analysis
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

When developers change a program, regression tests can fail not only due to faults in the program but also due to out-of-date test code that does not reflect the desired behavior of the program. When this occurs, it is necessary to repair test code such that the tests pass. Repairing tests manually is difficult and time consuming. We recently developed ReAssert, a tool that can automatically repair broken unit tests, but only if they lack complex control flow or operations on expected values. This paper introduces symbolic test repair, a technique based on symbolic execution, which can overcome some of ReAssert's limitations. We reproduce experiments from earlier work and find that symbolic test repair improves upon previously reported results both quantitatively and qualitatively. We also perform new experiments which confirm the benefits of symbolic test repair and also show surprising similarities in test failures for open-source Java and .NET programs. Our experiments use Pex, a powerful symbolic execution engine for .NET, and we find that Pex provides over half of the repairs possible from the theoretically ideal symbolic test repair.