Reporting computing projects through structured abstracts: a quasi-experiment

  • Authors:

  • David Budgen;Andy J. Burn;Barbara Kitchenham

  • Affiliations:

  • School of Engineering & Computing Sciences, Durham University, Durham, UK DH1 3LE;School of Engineering & Computing Sciences, Durham University, Durham, UK DH1 3LE;School of Computing & Maths, Keele University, Staffordshire, UK ST5 5BG

  • Venue:
  • Empirical Software Engineering
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Previous work has demonstrated that the use of structured abstracts can lead to greater completeness and clarity of information, making it easier for researchers to extract information about a study. In academic year 2007/08, Durham University's Computer Science Department revised the format of the project report that final year students were required to write, from a `traditional dissertation' format, using a conventional abstract, to that of a 20-page technical paper, together with a structured abstract. This study set out to determine whether inexperienced authors (students writing their final project reports for computing topics) find it easier to produce good abstracts, in terms of completeness and clarity, when using a structured form rather than a conventional form. We performed a controlled quasi-experiment in which a set of `judges' each assessed one conventional and one structured abstract for its completeness and clarity. These abstracts were drawn from those produced by four cohorts of final year students: two preceding the change, and the two following. The assessments were performed using a form of checklist that is similar to those used for previous experimental studies. We used 40 abstracts (10 per cohort) and 20 student `judges' to perform the evaluation. Scored on a scale of 0.1---1.0, the mean for completeness increased from 0.37 to 0.61 when using a structured form. For clarity, using a scale of 1---10, the mean score increased from 5.1 to 7.2. For a minimum goal of scoring 50% for both completeness and clarity, only 3 from 19 conventional abstracts achieved this level, while only 3 from 20 structured abstracts failed to reach it. We conclude that the use of a structured form for organising the material of an abstract can assist inexperienced authors with writing technical abstracts that are clearer and more complete than those produced without the framework provided by such a mechanism.