Towards a benchmark for traceability

  • Authors:
  • Eya Ben Charrada;David Caspar;Cédric Jeanneret;Martin Glinz

  • Affiliations:
  • Department of Informatics, University of Zurich, Zurich, Switzerland;Department of Informatics, University of Zurich, Zurich, Switzerland;Department of Informatics, University of Zurich, Zurich, Switzerland;Department of Informatics, University of Zurich, Zurich, Switzerland

  • Venue:
  • Proceedings of the 12th International Workshop on Principles of Software Evolution and the 7th annual ERCIM Workshop on Software Evolution
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Rigorously evaluating and comparing traceability link generation techniques is a challenging task. In fact, traceability is still expensive to implement and it is therefore difficult to find a complete case study that includes both a rich set of artifacts and traceability links among them. Consequently, researchers usually have to create their own case studies by taking a number of existing artifacts and creating traceability links for them. There are two major issues related to the creation of one's own example. First, creating a meaningful case study is time consuming. Second, the created case usually covers a limited set of artifacts and has a limited applicability (e.g., a case with traces from high-level requirements to low-level requirements cannot be used to evaluate traceability techniques that are meant to generate links from documentation to source code). We propose a benchmark for traceability that includes all artifacts that are typically produced during the development of a software system and with end-to-end traceability linking. The benchmark is based on an irrigation system that was elaborated in a book about software design. The main task considered by the benchmark is the generation of traceability links among different types of software artifacts. Such a traceability benchmark will help advance research in this field because it facilitates the evaluation and comparison of traceability techniques and makes the replication of experiments an easy task. As a proof of concept we used the benchmark to evaluate the precision and recall of a link generation technique based on the vector space model. Our results are comparable to those obtained by other researchers using the same technique.