Clustering test cases to achieve effective and scalable prioritisation incorporating expert knowledge

  • Authors:

  • Shin Yoo;Mark Harman;Paolo Tonella;Angelo Susi

  • Affiliations:

  • King's College London, London, United Kingdom;King's College London, London, United Kingdom;Fondazione Bruno Kessler, Trento, Italy;Fondazione Bruno Kessler, Trento, Italy

  • Venue:
  • Proceedings of the eighteenth international symposium on Software testing and analysis
  • Year:
  • 2009

Quantified Score

Hi-index 0.01

Visualization

Abstract

Pair-wise comparison has been successfully utilised in order to prioritise test cases by exploiting the rich, valuable and unique knowledge of the tester. However, the prohibitively large cost of the pair-wise comparison method prevents it from being applied to large test suites. In this paper, we introduce a cluster-based test case prioritisation technique. By clustering test cases, based on their dynamic runtime behaviour, we can reduce the required number of pair-wise comparisons significantly. The approach is evaluated on seven test suites ranging in size from 154 to 1,061 test cases. We present an empirical study that shows that the resulting prioritisation is more effective than existing coverage-based prioritisation techniques in terms of rate of fault detection. Perhaps surprisingly, the paper also demonstrates that clustering (even without human input) can outperform unclustered coverage-based technologies, and discusses an automated process that can be used to determine whether the application of the proposed approach would yield improvement.