Effective test metrics for test strategy evolution

  • Authors:

  • Yanping Chen;Robert L. Probert;Kyle Robeson

  • Affiliations:

  • School of Information Technology and Engineering, University of Ottawa, Canada;School of Information Technology and Engineering, University of Ottawa, Canada;Integration Test Team Lead, E-commerce Development, IBM Canada Ltd., Canada

  • Venue:
  • CASCON '04 Proceedings of the 2004 conference of the Centre for Advanced Studies on Collaborative research
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

How to measure a test process is a required competence for an effective software test manager. Effective test process measurement is essential for designing, evaluating and managing a cost-effective test strategy. Although some existing research has proposed many metrics for these purposes, most of those metrics are ignored or used individually. This paper summarizes a test process measurement pilot project of the IBM Electronic Commerce Development (ECD) test teams, and analyzes the effectiveness of using a "metric set" of complementary metrics for cost, time, and quality to measure the quality of test process based on the results of our project. Details are given for a "fictitious" project in order to explain the detailed considerations that go into adopting a cost-effective metric strategy. Further discussions about implementation problems, test wisdom learned in the project, and cost-effectiveness of the measurement activities and metrics are also presented. We conclude with suggestions for extensions of the work into the area of real-time, in-phase, in-process metrics.