Proceedings of the 27th International Conference on Very Large Data Bases
Observations and lessons learned from automated testing
Proceedings of the 27th international conference on Software engineering
Automatically repairing event sequence-based GUI test suites for regression testing
ACM Transactions on Software Engineering and Methodology (TOSEM)
Maintaining and evolving GUI-directed test scripts
ICSE '09 Proceedings of the 31st International Conference on Software Engineering
GUI testing using computer vision
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Software test automation in practice: empirical observations
Advances in Software Engineering - Special issue on software test automation
WATER: Web Application TEst Repair
Proceedings of the First International Workshop on End-to-End Test Script Engineering
Automated System Testing Using Visual GUI Testing Tools: A Comparative Study in Industry
ICST '12 Proceedings of the 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation
Proceedings of the 34th International Conference on Software Engineering
Efficiently scripting change-resilient tests
Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering
Hi-index | 0.00 |
Test automation, which involves the conversion of manual test cases to executable test scripts, is necessary to carry out efficient regression testing of GUI-based applications. However, test automation takes significant investment of time and skilled effort. Moreover, it is not a one-time investment: as the application or its environment evolves, test scripts demand continuous patching. Thus, it is challenging to perform test automation in a cost-effective manner. At IBM, we developed a tool, called ATA, to meet this challenge. ATA has novel features that are designed to lower the cost of initial test automation significantly. Moreover, ATA has the ability to patch scripts automatically for certain types of application or environment changes. How well does ATA meet its objectives in the real world? In this paper, we present a detailed case study in the context of a challenging production environment: an enterprise web application that has over 6500 manual test cases, comes in two variants, evolves frequently, and needs to be tested on multiple browsers in time-constrained and resource-constrained regression cycles. We measured how well ATA improved the efficiency in initial automation. We also evaluated the effectiveness of ATA's change-resilience along multiple dimensions: application versions, browsers, and browser versions. Our study highlights several lessons for test-automation practitioners as well as open research problems in test automation.