Schema-driven experiment management: declarative testing with dexterity

  • Authors:
  • Daniel J. Farrar

  • Affiliations:
  • Sybase, Waterloo, Ontario, Canada

  • Venue:
  • Proceedings of the Third International Workshop on Testing Database Systems
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

The testing of database management software to ensure correct, stable, and efficient operation requires the application not only of fixed regression test cases, but of ad hoc experiments to search for previously-unknown bugs and performance optimizations. This paper describes the Dexterity system, a declarative framework for the rapid description, execution, and documentation of such experiments, using a relational database. In this design, experimental specifications and results are stored in the database, and dependencies between the deliverables of each step of an experiment are modelled as foreign key relationships in the schema (with additional annotations to indicate the type and duration of the dependency relationship). Worker processes performing the testing tasks communicate with the server via stateless web services, and are scheduled by querying the data and schema of the database. This design concentrates the effort of the human experimenter into two phases: an experiment description phase, and a data analysis phase, while allowing (sometimes lengthy) test execution over a large number of dynamic independent variables to be conducted with minimal intervention.