Using observational pilot studies to test and improve lab packages

  • Authors:
  • Manoel Mendonça;Daniela Cruzes;Josemeire Dias;Maria Cristina Ferreira de Oliveira

  • Affiliations:
  • Salvador University (UNIFACS), Salvador, Brazil;Salvador University (UNIFACS), Salvador, Brazil;Salvador University (UNIFACS), Salvador, Brazil;Instituto de Ciências Matemáticas e de Computacão, São Carlos, Brazil

  • Venue:
  • Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

Controlled experiments are a key approach to evaluate and evolve our understanding of software engineering technologies. However, defining and running a controlled experiment is a difficult and errorprone task. This paper argues that one can significantly reduce the risks associated with defining a new controlled experiment by running a set of well-planned observational pilot studies aimed at improving the experimental material. It gives the steps for such an approach and illustrates it with a case study that shows the definition of an experiment to evaluate inspection techniques for information visualization tools through a set of four observational studies and one experimental trial. Based on the lessons learned we present some guidelines on how to test and improve experimental material in such way.