An experimental comparison of reading techniques for defect detection in UML design documents
Journal of Systems and Software
Applying meta-analytical procedures to software engineering experiments
Journal of Systems and Software
The Handbook of Research Synthesis
The Handbook of Research Synthesis
Meta Analysis—A Silver Bullet—for Meta-Analysts
Empirical Software Engineering
APSEC '00 Proceedings of the Seventh Asia-Pacific Software Engineering Conference
Experimenting with Error Abstraction in Requirements Documents
METRICS '98 Proceedings of the 5th International Symposium on Software Metrics
Research Synthesis in Software Engineering: A Case for Meta-Analysis
METRICS '99 Proceedings of the 6th International Symposium on Software Metrics
Quantitative Modeling of Software Reviews in an Industrial Setting
METRICS '99 Proceedings of the 6th International Symposium on Software Metrics
What We Have Learned About Fighting Defects
METRICS '02 Proceedings of the 8th International Symposium on Software Metrics
Combining data from reading experiments in software inspections: a feasibility study
Lecture notes on empirical software engineering
ISESE '03 Proceedings of the 2003 International Symposium on Empirical Software Engineering
Software Reviews: The State of the Practice
IEEE Software
Investigating the Active Guidance Factor in Reading Techniques for Defect Detection
ISESE '04 Proceedings of the 2004 International Symposium on Empirical Software Engineering
Towards Evidence in Software Engineering
ISESE '04 Proceedings of the 2004 International Symposium on Empirical Software Engineering
Assessing the Impact of Active Guidance for Defect Detection: A Replicated Experiment
METRICS '04 Proceedings of the Software Metrics, 10th International Symposium
Accumulation and presentation of empirical evidence: problems and challenges
REBSE '05 Proceedings of the 2005 workshop on Realising evidence-based software engineering
Perspective-Based Reading: A Replicated Experiment Focused on Individual Reviewer Effectiveness
Empirical Software Engineering
ACM SIGSOFT Software Engineering Notes
Systematic literature reviews in software engineering - A tertiary study
Information and Software Technology
Evaluating a model of software managers' information needs: an experiment
Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement
An ecological perspective towards the evolution of quantitative studies in software engineering
Proceedings of the 17th International Conference on Evaluation and Assessment in Software Engineering
Empirical Software Engineering
Hi-index | 0.00 |
One of the main challenges in empirical software engineering today lies in the aggregation of evidence. Existing summaries often use qualitative narrative approaches or ad-hoc quantitative methods, such as box plots. With these, information important for decision makers, such as existence and magnitude of a technology's effect, is hard to obtain objectively. Meta-analysis addresses this issue by providing objective quantitative information about a set of studies; however, its usefulness for software engineering studies suffers from high heterogeneity of the studies and missing information. In this paper, we describe an approach for quantitative aggregation of controlled experiments that reduces these two problems. We demonstrate the approach by aggregating available experiments to investigate whether Perspective-Based reading (PBR) improves team effectiveness compared to alternative reading approaches. We then compare the results of our aggregation to previous summaries addressing PBR's team effectiveness. Although the findings are similar, our approach is able to provide the required quantitative information objectively. Our aggregation showed that there is no clear positive effect of PBR: Inspection teams using PBR on requirements documents are more effective when compared to ad-hoc approaches, but are less effective when compared to checklists. In addition, we found strong indicators of researcher bias.