Building Knowledge through Families of Experiments
IEEE Transactions on Software Engineering
Experimentation in software engineering: an introduction
Experimentation in software engineering: an introduction
Extreme programming examined
Case study: extreme programming in a university environment
ICSE '01 Proceedings of the 23rd International Conference on Software Engineering
Test Driven Development: By Example
Test Driven Development: By Example
Empirical Software Engineering
Empirical Software Engineering
Hints for Reviewing Empirical Work in Software Engineering
Empirical Software Engineering
Perceptions of Agile Practices: A Student Survey
Proceedings of the Second XP Universe and First Agile Universe Conference on Extreme Programming and Agile Methods - XP/Agile Universe 2002
Empirical Findings in Agile Methods
Proceedings of the Second XP Universe and First Agile Universe Conference on Extreme Programming and Agile Methods - XP/Agile Universe 2002
Assessing test-driven development at IBM
Proceedings of the 25th International Conference on Software Engineering
A Prototype Empirical Evaluation of Test Driven Development
METRICS '04 Proceedings of the Software Metrics, 10th International Symposium
On the Effectiveness of the Test-First Approach to Programming
IEEE Transactions on Software Engineering
On the Influence of Test-Driven Development on Software Design
CSEET '06 Proceedings of the 19th Conference on Software Engineering Education & Training
Proceedings of the 2006 international workshop on Software quality
Evaluating the efficacy of test-driven development: industrial case studies
Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
Evaluating advantages of test driven development: a controlled experiment with professionals
Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
Refactoring--Does It Improve Software Quality?
WoSQ '07 Proceedings of the 5th International Workshop on Software Quality
ESEM '07 Proceedings of the First International Symposium on Empirical Software Engineering and Measurement
An Experimental Evaluation of the Effectiveness and Efficiency of the Test Driven Development
ESEM '07 Proceedings of the First International Symposium on Empirical Software Engineering and Measurement
IEEE Transactions on Software Engineering
The effect of experience on the test-driven development process
Empirical Software Engineering
Test-driven learning in early programming courses
Proceedings of the 39th SIGCSE technical symposium on Computer science education
The role of replications in Empirical Software Engineering
Empirical Software Engineering
Comparing design and code metrics for software quality prediction
Proceedings of the 4th international workshop on Predictor models in software engineering
A survey of evidence for test-driven development in academia
ACM SIGCSE Bulletin
Empirical investigation towards the effectiveness of Test First programming
Information and Software Technology
Towards traceable test-driven development
TEFSE '09 Proceedings of the 2009 ICSE Workshop on Traceability in Emerging Forms of Software Engineering
Proceedings of the 2005 conference on Software Engineering: Evolution and Emerging Technologies
Evaluating Test-Driven Development in an Industry-Sponsored Capstone Project
ITNG '09 Proceedings of the 2009 Sixth International Conference on Information Technology: New Generations
Information and Software Technology
Operational definition and automated inference of test-driven development with Zorro
Automated Software Engineering
A checklist for integrating student empirical studies with research and teaching goals
Empirical Software Engineering
Test-Driven Development: An Empirical Evaluation of Agile Practice
Test-Driven Development: An Empirical Evaluation of Agile Practice
Five challenges in teaching XP
XP'03 Proceedings of the 4th international conference on Extreme programming and agile processes in software engineering
Challenges in teaching test driven development
XP'03 Proceedings of the 4th international conference on Extreme programming and agile processes in software engineering
PROFES'06 Proceedings of the 7th international conference on Product-Focused Software Process Improvement
Lessons learned from an XP experiment with students: test-first needs more teachings
PROFES'06 Proceedings of the 7th international conference on Product-Focused Software Process Improvement
Information and Software Technology
An ecological perspective towards the evolution of quantitative studies in software engineering
Proceedings of the 17th International Conference on Evaluation and Assessment in Software Engineering
Considering rigor and relevance when evaluating test driven development: A systematic review
Information and Software Technology
On the role of tests in test-driven development: a differentiated and partial replication
Empirical Software Engineering
Hi-index | 0.00 |
Context: Test-driven development is an approach to software development, where automated tests are written before production code in highly iterative cycles. Test-driven development attracts attention as well as followers in professional environment; however empirical evidence of its superiority regarding its effect on productivity, code and tests compared to test-last development is still fairly limited. Moreover, it is not clear if the supposed benefits come from writing tests before code or maybe from high iterativity/short development cycles. Objective: This paper describes a family of controlled experiments comparing test-driven development to micro iterative test-last development with emphasis on productivity, code properties (external quality and complexity) and tests (code coverage and fault-finding capabilities). Method: Subjects were randomly assigned to test-driven and test-last groups. Controlled experiments were conducted for two years, in an academic environment and in different developer contexts (pair programming and individual programming contexts). Number of successfully implemented stories, percentage of successful acceptance tests, McCabe's code complexity, code coverage and mutation score indicator were measured. Results: Experimental results and their selective meta-analysis show no statistically significant differences between test-driven development and iterative test-last development regarding productivity (@g^2(6)=4.799, p=1.0, r=.107, 95% CI (confidence interval): -.149 to .349), code complexity (@g^2(6)=8.094, p=.46, r=.048, 95% CI: -.254 to .341), branch coverage (@g^2(6)=13.996, p=.059, r=.182, 95% CI: -.081 to .421), percentage of acceptance tests passed (one experiment, Mann-Whitney U=125.0, p=.98, r=.066) and mutation score indicator (@g^2(4)=3.807, p=.87, r=.128, 95% CI: -.162 to .398). Conclusion: According to our findings, the benefits of test-driven development compared to iterative test-last development are small and thus in practice relatively unimportant, although effects are positive. There is an indication of test-driven development endorsing better branch coverage, but effect size is considered small.