Discrepancy discovery in search-enhanced testing

  • Authors:

  • Werner Janjic;Florian Barth;Oliver Hummel;Colin Atkinson

  • Affiliations:

  • University of Mannheim, Mannheim, Germany;University of Mannheim, Mannheim, Germany;University of Mannheim, Mannheim, Germany;University of Mannheim, Mannheim, Germany

  • Venue:
  • Proceedings of the 3rd International Workshop on Search-Driven Development: Users, Infrastructure, Tools, and Evaluation
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Automating software testing can significantly reduce the time and effort required to assure the quality of software systems, and over recent years significant strides have been made in test automation techniques. However, one aspect of software testing that has always resisted full automation is the determination of the expected results for given system states and input values - the so called "oracle problem". Fortunately, the recent advent of a new generation of software search engines containing millions of reusable software artifacts offers an elegant solution to this dilemma. Once a search engine is able to deliver multiple results that conform to a given specification (by searching for and adapting preexisting components), multi-version testing of software with "harvested" oracles becomes a feasible alternative to manual oracle definition. In this paper we present an approach to Search-Enhanced Testing with a focus on the discovery of discrepancies between the results returned by harvested test oracles and a Component Under Test for randomly generated test invocations. Our current research focuses on validating the hypothesis that human test engineers will find more defects when analyzing such automatically discovered discrepancies than when developing test cases using traditional coverage criteria.