Semi-automatic update of applications in response to library changes
ICSM '96 Proceedings of the 1996 International Conference on Software Maintenance
Proceedings of the 9th European software engineering conference held jointly with 11th ACM SIGSOFT international symposium on Foundations of software engineering
CatchUp!: capturing and replaying refactorings to support API evolution
Proceedings of the 27th international conference on Software engineering
Facilitating software evolution research with kenyon
Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering
How do APIs evolve? A story of refactoring: Research Articles
Journal of Software Maintenance and Evolution: Research and Practice - IEEE International Conference on Software Maintenance (ICSM2005)
Recommending adaptive changes for framework evolution
Proceedings of the 30th international conference on Software engineering
Automated Session Data Repair for Web Application Regression Testing
ICST '08 Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation
Automatically repairing event sequence-based GUI test suites for regression testing
ACM Transactions on Software Engineering and Methodology (TOSEM)
Automatically finding patches using genetic programming
ICSE '09 Proceedings of the 31st International Conference on Software Engineering
Maintaining and evolving GUI-directed test scripts
ICSE '09 Proceedings of the 31st International Conference on Software Engineering
Test-Suite Augmentation for Evolving Software
ASE '08 Proceedings of the 2008 23rd IEEE/ACM International Conference on Automated Software Engineering
ReAssert: Suggesting Repairs for Broken Unit Tests
ASE '09 Proceedings of the 2009 IEEE/ACM International Conference on Automated Software Engineering
Repairing GUI Test Suites Using a Genetic Algorithm
ICST '10 Proceedings of the 2010 Third International Conference on Software Testing, Verification and Validation
Automated fixing of programs with contracts
Proceedings of the 19th international symposium on Software testing and analysis
On test repair using symbolic execution
Proceedings of the 19th international symposium on Software testing and analysis
A graph-based approach to API usage adaptation
Proceedings of the ACM international conference on Object oriented programming systems languages and applications
Proceedings of the 33rd International Conference on Software Engineering
Specification-based program repair using SAT
TACAS'11/ETAPS'11 Proceedings of the 17th international conference on Tools and algorithms for the construction and analysis of systems: part of the joint European conferences on theory and practice of software
Systematic editing: generating program transformations from an example
Proceedings of the 32nd ACM SIGPLAN conference on Programming language design and implementation
WATER: Web Application TEst Repair
Proceedings of the First International Workshop on End-to-End Test Script Engineering
Synthesizing method sequences for high-coverage testing
Proceedings of the 2011 ACM international conference on Object oriented programming systems languages and applications
Symstra: a framework for generating object-oriented unit tests using symbolic execution
TACAS'05 Proceedings of the 11th international conference on Tools and Algorithms for the Construction and Analysis of Systems
Supporting Test Suite Evolution through Test Case Adaptation
ICST '12 Proceedings of the 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation
Automatic patch generation learned from human-written patches
Proceedings of the 2013 International Conference on Software Engineering
TestEvol: a tool for analyzing test-suite evolution
Proceedings of the 2013 International Conference on Software Engineering
Strategies for avoiding text fixture smells during software evolution
Proceedings of the 10th Working Conference on Mining Software Repositories
Proceedings of the 12th international conference on Generative programming: concepts & experiences
Is this a bug or an obsolete test?
ECOOP'13 Proceedings of the 27th European conference on Object-Oriented Programming
Hi-index | 0.00 |
Test suites, once created, rarely remain static. Just like the application they are testing, they evolve throughout their lifetime. Test obsolescence is probably the most known reason for test-suite evolution---test cases cease to work because of changes in the code and must be suitably repaired. Repairing existing test cases manually, however, can be extremely time consuming, especially for large test suites, which has motivated the recent development of automated test-repair techniques. We believe that, for developing effective repair techniques that are applicable in real-world scenarios, a fundamental prerequisite is a thorough understanding of how test cases evolve in practice. Without such knowledge, we risk to develop techniques that may work well for only a small number of tests or, worse, that may not work at all in most realistic cases. Unfortunately, to date there are no studies in the literature that investigate how test suites evolve. To tackle this problem, in this paper we present a technique for studying test-suite evolution, a tool that implements the technique, and an extensive empirical study in which we used our technique to study many versions of six real-world programs and their unit test suites. This is the first study of this kind, and our results reveal several interesting aspects of test-suite evolution. In particular, our findings show that test repair is just one possible reason for test-suite evolution, whereas most changes involve refactorings, deletions, and additions of test cases. Our results also show that test modifications tend to involve complex, and hard-to-automate, changes to test cases, and that existing test-repair techniques that focus exclusively on assertions may have limited practical applicability. More generally, our findings provide initial insight on how test cases are added, removed, and modified in practice, and can guide future research efforts in the area of test-suite evolution.