Test confessions: a study of testing practices for plug-in systems

  • Authors:
  • Michaela Greiler;Arie van Deursen;Margaret-Anne Storey

  • Affiliations:
  • TU Delft, Netherlands;TU Delft, Netherlands;University of Victoria, Canada

  • Venue:
  • Proceedings of the 34th International Conference on Software Engineering
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

Testing plug-in based systems is challenging due to complex interactions among many different plug-ins, and variations in version and configuration. The objective of this paper is to find out how developers address this test challenge. To that end, we conduct a qualitative (grounded theory) study, in which we interview 25 senior practitioners about how they test plug-ins and applications built on top of the Eclipse plug-in framework. The outcome is an overview of the testing practices currently used, a set of identified barriers limiting the adoption of test practices, and an explanation of how limited testing is compensated by self-hosting of projects and by involving the community. These results are supported by a structured survey of more than 150 professionals. The study reveals that unit testing plays a key role, whereas plug-in specific integration problems are identified and resolved by the community. Based on our findings, we propose a series of recommendations and areas for future research.