Reconciling software development models: A quasi-systematic review
Journal of Systems and Software
Systematic literature studies: database searches vs. backward snowballing
Proceedings of the ACM-IEEE international symposium on Empirical software engineering and measurement
25 years of software engineering in Brazil: Beyond an insider's view
Journal of Systems and Software
A tertiary study: experiences of conducting systematic literature reviews in software engineering
Proceedings of the 17th International Conference on Evaluation and Assessment in Software Engineering
An ecological perspective towards the evolution of quantitative studies in software engineering
Proceedings of the 17th International Conference on Evaluation and Assessment in Software Engineering
A systematic review of systematic review process research in software engineering
Information and Software Technology
Critical success factors taxonomy for software process deployment
Software Quality Control
| Hi-index | 0.01 |
BACKGROUND—The systematic review is becoming a more commonly employed research instrument in empirical software engineering. Before undue reliance is placed on the outcomes of such reviews it would seem useful to consider the robustness of the approach in this particular research context. OBJECTIVE—The aim of this study is to assess the reliability of systematic reviews as a research instrument. In particular, we wish to investigate the consistency of process and the stability of outcomes. METHOD—We compare the results of two independent reviews undertaken with a common research question. RESULTS—The two reviews find similar answers to the research question, although the means of arriving at those answers vary. CONCLUSIONS—In addressing a well-bounded research question, groups of researchers with similar domain experience can arrive at the same review outcomes, even though they may do so in different ways. This provides evidence that, in this context at least, the systematic review is a robust research method.