Developing search strategies for detecting relevant experiments

  • Authors:

  • Oscar Dieste;Anna Grimán;Natalia Juristo

  • Affiliations:

  • Facultad de Informática, Universidad Politénica de Madrid, Boadilla del Monte, Spain 28660;Universidad Simón Bolívar, Caracas, Venezuela 1080A;Facultad de Informática, Universidad Politénica de Madrid, Boadilla del Monte, Spain 28660

  • Venue:
  • Empirical Software Engineering
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

Our goal is to analyze the optimality of search strategies for use in systematic reviews of software engineering experiments. Studies retrieval is an important problem in any evidence-based discipline. This question has not been examined for evidence-based software engineering as yet. We have run several searches exercising different terms denoting experiments to evaluate their recall and precision. Based on our evaluation, we propose using a high recall strategy when there are plenty of resources or the results need to be exhaustive. For any other case, we propose optimal, or even acceptable, search strategies. As a secondary goal, we have analysed trends and weaknesses in terminology used in articles reporting software engineering experiments. We have found that it is impossible for a search strategy to retrieve 100% of the experiments of interest (as happens in other experimental disciplines), because of the shortage of reporting standards in the community.