Searching for Cognitively Diverse Tests: Towards Universal Test Diversity Metrics

  • Authors:
  • Robert Feldt;Richard Torkar;Tony Gorschek;Wasif Afzal

  • Affiliations:
  • -;-;-;-

  • Venue:
  • ICSTW '08 Proceedings of the 2008 IEEE International Conference on Software Testing Verification and Validation Workshop
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Search-based software testing (SBST) has shown a po- tential to decrease cost and increase quality of testing- related software development activities. Research in SBST has so far mainly focused on the search for isolated tests that are optimal according to a fitness function that guides the search. In this paper we make the case for fitness func- tions that measure test fitness in relation to existing or pre- viously found tests; a test is good if it is diverse from other tests. We present a model for test variability and propose the use of a theoretically optimal diversity metric at vari- ation points in the model. We then describe how to apply a practically useful approximation to the theoretically opti- mal metric. The metric is simple and powerful and can be adapted to a multitude of different test diversity measure- ment scenarios. We present initial results from an experi- ment to compare how similar to human subjects, the metric can cluster a set of test cases. To carry out the experiment we have extended an existing framework for test automation in an object-oriented, dynamic programming language.