Accumulation and presentation of empirical evidence: problems and challenges
REBSE '05 Proceedings of the 2005 workshop on Realising evidence-based software engineering
Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
PROFES '08 Proceedings of the 9th international conference on Product-Focused Software Process Improvement
ESEM '09 Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement
Adapting PROFES for use in an agile process: an industry experience report
PROFES'05 Proceedings of the 6th international conference on Product Focused Software Process Improvement
Hi-index | 0.00 |
The aggregation of studies is of growing interest for the Empirical Software Engineering community, since the numbers of studies steadily grow. In this paper we discuss challenges with the aggregation of studies into a common body of knowledge, based on a quantitative and qualitative evaluation of experience from the Experimental Software Engineering Network, ESERNET. Challenges are that the number of studies available is usually low, and the studies that exist are often too scattered and diverse to allow systematic aggregation as a means for generating evidence. ESERNET therefore attempted to coordinate studies and thus create research synergies to achieve a sufficiently large number of comparable studies to allow for aggregation; however, the coordination approach of ESERNET proved to be insufficient. Based on some lessons learned from ESERNET, a four-step procedure for evolving Empirical Software Engineering towards the generation of evidence is proposed. This consists of (1) developing a methodology for aggregating different kinds of empirical results, (2) establishing guidelines for performing, analyzing, and reporting studies as well as for aggregating the results for every kind of empirical study, (3) extract evidence, that is, apply the methodology to different areas of software engineering, and (4) package the extracted evidence into guidelines for practice.