Case Studies for Method and Tool Evaluation

  • Authors:

  • Barbara Kitchenham;Lesley Pickard;Shari Lawrence Pfleeger

  • Affiliations:

  • -;-;-

  • Venue:
  • IEEE Software
  • Year:
  • 1995

Quantified Score

Hi-index 0.00

Visualization

Abstract

The last decade has seen explosive growth in the number of software-engineering methods and tools, each one offering to improve some characteristic of software, its development, or its maintenance. With an increasing awareness of the competitive advantage to be gained from continuing process improvement, we all seek methods and tools that will make us more productive and improve the quality of our software. But how do we ensure that our changes lead to positive improvement? Suppose you have decided to evaluate a technology. How do you proceed? Do you do a survey? An experiment? A case study? In this article, we discuss the conditions under which each type of investigation is appropriate. Then, because good case studies are as rare as they are powerful and informative, we focus on how to do a proper and effective case study. Although they cannot achieve the scientific rigor of formal experiments, case studies can provide sufficient information to help you judge if specific technologies will benefit your own organization or project. Even when you cannot do a case study of your own, the principles of good case-study analysis will help you determine if the case-study results you read about are applicable to your situation. Good case studies involve specifying the hypothesis under test; using state variables for project selection and data analysis; establishing a basis for comparisons; planning case studies properly; and using appropriate presentation and analysis techniques to assess the results.