A genetic approach for random testing of database systems

  • Authors:

  • Hardik Bati;Leo Giakoumakis;Steve Herbert;Aleksandras Surna

  • Affiliations:

  • Microsoft Corporation, One Microsoft Way, Redmond, WA;Microsoft Corporation, One Microsoft Way, Redmond, WA;Microsoft Corporation, One Microsoft Way, Redmond, WA;Microsoft Corporation, One Microsoft Way, Redmond, WA

  • Venue:
  • VLDB '07 Proceedings of the 33rd international conference on Very large data bases
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

Testing a database engine has been and continues to be a challenging task. The space of possible SQL queries along with their possible access paths is practically unbounded. Moreover, this space is continuously increasing in size as the feature set of modern DBMS systems expands with every product release. To tackle these problems, random query generator tools have been used to create large numbers of test cases. While such test case generators enable the creation of complex and syntactically correct SQL queries, they do not guarantee that the queries produced return results or exercise desired DBMS components. Very often the generated queries contain logical contradictions, which cause "short-circuits" at the query optimization layer, failing to exercise the lower layers of the database engine (query optimization, query execution, access methods, etc.) In this paper we present a random test case generation technique, which provides solutions to the above problems. Our technique utilizes execution feedback, obtained from the DBMS under test, in order to guide the test generation process toward specific DBMS subcomponents and rarely exercised code paths. Test cases are created incrementally using a genetic approach, which synthesizes query characteristics that are of interest for the purposes of test coverage. Our experiments indicate that our technique can outperform other methods of random testing in terms of efficiency and code coverage. We also provide experimental results which show that the use of execution feedback improves code coverage of specific DBMS components. Finally, we share our experiences gained from using this testing approach during the development cycles of Microsoft SQL Server.