A Survey of Controlled Experiments in Software Engineering
IEEE Transactions on Software Engineering
Evaluating guidelines for empirical software engineering studies
Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
Journal of Systems and Software
REBSE '07 Proceedings of the Second International Workshop on Realising Evidence-Based Software Engineering
Reporting computing projects through structured abstracts: a quasi-experiment
Empirical Software Engineering
Systematic mapping studies in software engineering
EASE'08 Proceedings of the 12th international conference on Evaluation and Assessment in Software Engineering
A follow-up empirical evaluation of evidence based software engineering by undergraduate students
EASE'08 Proceedings of the 12th international conference on Evaluation and Assessment in Software Engineering
Hi-index | 0.00 |
CONTEXT: Systematic literature reviews largely rely upon using the titles and abstracts of primary studies as the basis for determining their relevance. However, our experience indicates that the abstracts for software engineering papers are frequently of such poor quality they cannot be used to determine the relevance of papers. Both medicine and psychology recommend the use of structured abstracts to improve the quality of abstracts. AIM: This study investigates whether structured abstracts are more complete and easier to understand than non-structured abstracts for software engineering papers that describe experiments. METHOD: We constructed structured abstracts for a random selection of 25 papers describing software engineering experiments. The original abstract was assessed for clarity (assessed subjectively on a scale of 1 to 10) and completeness (measured with a questionnaire of 18 items) by the researcher who constructed the structured version. The structured abstract was reviewed for clarity and completeness by another member of the research team. We used a paired 't' test to compare the word length, clarity and completeness of the original and structured abstracts. RESULTS: The structured abstracts were significantly longer than the original abstracts (size difference =106.4 words with 95% confidence interval 78.1 to 134.7). However, the structured abstracts had a higher clarity score (clarity difference= 1.47 with 95% confidence interval 0.47 to 2.41) and were more complete (completeness difference=3.39 with 95% confidence intervals 4.76 to 7.56). CONCLUSIONS: The results of this study are consistent with previous research on structured abstracts. However, in this study, the subjective estimates of completeness and clarity were made by the research team. Future work will solicit assessments of the structured and original abstracts from independent sources (students and researchers).