Experimentation in software engineering: an introduction
Experimentation in software engineering: an introduction
Test Driven Development: By Example
Test Driven Development: By Example
Test Driven development: A Practical Guide
Test Driven development: A Practical Guide
An initial investigation of test driven development in industry
Proceedings of the 2003 ACM symposium on Applied computing
A cross-program investigation of students' perceptions of agile methods
Proceedings of the 27th international conference on Software engineering
On the Effectiveness of the Test-First Approach to Programming
IEEE Transactions on Software Engineering
Adopting XP practices for teaching object oriented programming
ACE '06 Proceedings of the 8th Australasian Conference on Computing Education - Volume 52
A Leveled Examination of Test-Driven Development Acceptance
ICSE '07 Proceedings of the 29th international conference on Software Engineering
On the Sustained Use of a Test-Driven Development Practice at IBM
AGILE '07 Proceedings of the AGILE 2007
Automated Recognition of Test-Driven Development with Zorro
AGILE '07 Proceedings of the AGILE 2007
The effect of experience on the test-driven development process
Empirical Software Engineering
Using differences among replications of software engineering experiments to gain knowledge
ESEM '09 Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement
Test-Driven Development: An Empirical Evaluation of Agile Practice
Test-Driven Development: An Empirical Evaluation of Agile Practice
To test before or to test after---an experimental investigation of the impact of test driven development
What Do We Know about Test-Driven Development?
IEEE Software
Impact of test-driven development on productivity, code and tests: A controlled experiment
Information and Software Technology
Lessons learned from an XP experiment with students: test-first needs more teachings
PROFES'06 Proceedings of the 7th international conference on Product-Focused Software Process Improvement
The Effects of Test-Driven Development on External Quality and Productivity: A Meta-Analysis
IEEE Transactions on Software Engineering
Hi-index | 0.00 |
Background: Test-Driven Development (TDD) is claimed to have positive effects on external code quality and programmers' productivity. The main driver for these possible improvements is the tests enforced by the test-first nature of TDD as previously investigated in a controlled experiment (i.e. the original study). Aim: Our goal is to examine the nature of the relationship between tests and external code quality as well as programmers' productivity in order to verify/ refute the results of the original study. Method: We conducted a differentiated and partial replication of the original setting and the related analyses, with a focus on the role of tests. Specifically, while the original study compared test-first vs. test-last, our replication employed the test-first treatment only. The replication involved 30 students, working in pairs or as individuals, in the context of a graduate course, and resulted in 16 software artifacts developed. We performed linear regression to test the original study's hypotheses, and analyses of covariance to test the additional hypotheses imposed by the changes in the replication settings. Results: We found significant correlation (Spearman coefficient = 0.66, with p-value = 0.004) between the number of tests and productivity, and a positive regression coefficient (p-value = 0.011). We found no significant correlation (Spearman coefficient = 0.41 with p-value = 0.11) between the number of tests and external code quality (regression coefficient p-value = 0.0513). For both cases we observed no statistically significant interaction caused by the subject units being individuals or pairs. Further, our results are consistent with the original study although there were changes in the timing constraints for finishing the task and the enforced development processes. Conclusions: This replication study confirms the results of the original study concerning the relationship between the number of tests vs. external code quality and programmer productivity. Moreover, this replication allows us to identify additional context variables, for which the original results still hold; namely the subject unit, timing constraint and isolation of test-first process. Based on our findings, we recommend practitioners to implement as many tests as possible in order to achieve higher baselines for quality and productivity.