Mutatis mutandis: evaluating DBMS test adequacy with mutation testing

  • Authors:
  • Ivan T. Bowman

  • Affiliations:
  • SAP AG

  • Venue:
  • Proceedings of the Sixth International Workshop on Testing Database Systems
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

Testing consumes significant human and machine resources, especially for large, complex systems such as database servers. While a variety of testing approaches have been proposed to improve the efficiency of the testing process, it is difficult to evaluate these approaches. Mutation testing has been proposed as a way to assess the adequacy of a test suite, assigning a score that can be used to compare testing approaches. While promising, serious obstacles appear to prevent mutation testing with large software systems. Recent advances in mutation testing have scaled to medium-sized programs of around 100,000 lines of code but to our knowledge there are no reported studies working with large systems with millions of lines of code and other features of database systems that complicate testing. In this paper we explore using mutation testing on a database server to evaluate its suitability for comparing test suites or testing approaches.