Building Knowledge through Families of Experiments
IEEE Transactions on Software Engineering
Applying meta-analytical procedures to software engineering experiments
Journal of Systems and Software
Knowledge-Sharing Issues in Experimental Software Engineering
Empirical Software Engineering
Analysis of the influence of communication between researchers on experiment replication
Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
A family of experiments to evaluate a functional size measurement procedure for Web applications
Journal of Systems and Software
Overcoming the First Hurdle: Why Organizations Do Not Adopt CMMI
ICSP '09 Proceedings of the International Conference on Software Process: Trustworthy Software Development Processes
Using differences among replications of software engineering experiments to gain knowledge
ESEM '09 Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement
A checklist for integrating student empirical studies with research and teaching goals
Empirical Software Engineering
1st International Workshop on Replication in Empirical Software Engineering Research (RESER)
Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2
Designing motivation strategies for software engineering teams: an empirical study
Proceedings of the 2010 ICSE Workshop on Cooperative and Human Aspects of Software Engineering
ACM SIGSOFT Software Engineering Notes
Replications types in experimental disciplines
Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement
Graphical versus textual software measurement modelling: an empirical study
Software Quality Control
Impact of test-driven development on productivity, code and tests: A controlled experiment
Information and Software Technology
The role of non-exact replications in software engineering experiments
Empirical Software Engineering
Program slicing-based cohesion measurement: the challenges of replicating studies using metrics
Proceedings of the 2nd International Workshop on Emerging Trends in Software Metrics
Architecture views illustrating the service automation aspect of SOA
Service research challenges and solutions for the future internet
Evaluating requirements modeling methods based on user perceptions: A family of experiments
Information Sciences: an International Journal
An industrial case study of classifier ensembles for locating software defects
Software Quality Control
CEE-SET'08 Proceedings of the Third IFIP TC 2 Central and East European conference on Software engineering techniques
ACM SIGSOFT Software Engineering Notes
Empirical Software Engineering
Empirical Software Engineering
Replication of software engineering experiments
Empirical Software Engineering and Verification
A note on rigour and replicability
ACM SIGSOFT Software Engineering Notes
Empirical validation of a usability inspection method for model-driven Web development
Journal of Systems and Software
Replicating mining studies with SOFAS
Proceedings of the 10th Working Conference on Mining Software Repositories
The value of design rationale information
ACM Transactions on Software Engineering and Methodology (TOSEM) - In memoriam, fault detection and localization, formal methods, modeling and design
On the impact of UML analysis models on source-code comprehensibility and modifiability
ACM Transactions on Software Engineering and Methodology (TOSEM)
Information and Software Technology
Combining service-orientation and software product line engineering: A systematic mapping study
Information and Software Technology
Effective data warehouse for information delivery: a literature survey and classification
International Journal of Networking and Virtual Organisations
ACM Computing Surveys (CSUR)
ACM SIGSOFT Software Engineering Notes
Empirical Software Engineering
Empirical Software Engineering
Replications of software engineering experiments
Empirical Software Engineering
Hi-index | 0.00 |
Replications play a key role in Empirical Software Engineering by allowing the community to build knowledge about which results or observations hold under which conditions. Therefore, not only can a replication that produces similar results as the original experiment be viewed as successful, but a replication that produce results different from those of the original experiment can also be viewed as successful. In this paper we identify two types of replications: exact replications, in which the procedures of an experiment are followed as closely as possible; and conceptual replications, in which the same research question is evaluated by using a different experimental procedure. The focus of this paper is on exact replications. We further explore them to identify two sub-categories: dependent replications, where researchers attempt to keep all the conditions of the experiment the same or very similar and independent replications, where researchers deliberately vary one or more major aspects of the conditions of the experiment. We then discuss the role played by each type of replication in terms of its goals, benefits, and limitations. Finally, we highlight the importance of producing adequate documentation for an experiment (original or replication) to allow for replication. A properly documented replication provides the details necessary to gain a sufficient understanding of the study being replicated without requiring the replicator to slavishly follow the given procedures.