Software testing based on formal specifications: a theory and a tool
Software Engineering Journal
Towards a method of programming with assertions
ICSE '92 Proceedings of the 14th international conference on Software engineering
Specification-based test oracles for reactive systems
ICSE '92 Proceedings of the 14th international conference on Software engineering
An integrated general purpose automated test environment
ISSTA '93 Proceedings of the 1993 ACM SIGSOFT international symposium on Software testing and analysis
Generating a test oracle from program documentation: work in progress
ISSTA '94 Proceedings of the 1994 ACM SIGSOFT international symposium on Software testing and analysis
TAOS: Testing with Analysis and Oracle Support
ISSTA '94 Proceedings of the 1994 ACM SIGSOFT international symposium on Software testing and analysis
TOBAC: a test case browser for testing object-oriented software
ISSTA '94 Proceedings of the 1994 ACM SIGSOFT international symposium on Software testing and analysis
The ASTOOT approach to testing object-oriented programs
ACM Transactions on Software Engineering and Methodology (TOSEM)
Oracles for checking temporal properties of concurrent systems
SIGSOFT '94 Proceedings of the 2nd ACM SIGSOFT symposium on Foundations of software engineering
ACM Transactions on Computer-Human Interaction (TOCHI)
Generating oracles from your favorite temporal logic specifications
SIGSOFT '96 Proceedings of the 4th ACM SIGSOFT symposium on Foundations of software engineering
Specification-based testing of reactive software: tools and experiments: experience report
ICSE '97 Proceedings of the 19th international conference on Software engineering
Software unit test coverage and adequacy
ACM Computing Surveys (CSUR)
A visual test development environment for GUI systems
Proceedings of the 1998 ACM SIGSOFT international symposium on Software testing and analysis
Using Test Oracles Generated from Program Documentation
IEEE Transactions on Software Engineering
Lutess: a specification-driven testing environment for synchronous software
Proceedings of the 21st international conference on Software engineering
Experimentation in software engineering: an introduction
Experimentation in software engineering: an introduction
Automatically Checking an Implementation against Its Formal Specification
IEEE Transactions on Software Engineering
Automated test oracles for GUIs
SIGSOFT '00/FSE-8 Proceedings of the 8th ACM SIGSOFT international symposium on Foundations of software engineering: twenty-first century applications
Data Abstraction, Implementation, Specification, and Testing
ACM Transactions on Programming Languages and Systems (TOPLAS)
An empirical study of regression test selection techniques
ACM Transactions on Software Engineering and Methodology (TOSEM)
Hierarchical GUI Test Case Generation Using Automated Planning
IEEE Transactions on Software Engineering - Special issue on 1999 international conference on software engineering
Coverage criteria for GUI testing
Proceedings of the 8th European software engineering conference held jointly with 9th ACM SIGSOFT international symposium on Foundations of software engineering
ACM SIGSOFT Software Engineering Notes
Experience in Testing the Motif Interface
IEEE Software
Specifying Concurrent Systems with TSL
IEEE Software
Formal Specifications and Test: Correctness and Oracle
Selected papers from the 11th Workshop on Specification of Abstract Data Types Joint with the 8th COMPASS Workshop on Recent Trends in Data Type Specification
A Simple and Practical Approach to Unit Testing: The JML and JUnit Way
ECOOP '02 Proceedings of the 16th European Conference on Object-Oriented Programming
Improving web application testing with user session data
Proceedings of the 25th International Conference on Software Engineering
User-Based Testing of GUI Sequences and Their Interactions
ISSRE '01 Proceedings of the 12th International Symposium on Software Reliability Engineering
Proceedings of the 9th European software engineering conference held jointly with 11th ACM SIGSOFT international symposium on Foundations of software engineering
GUI Ripping: Reverse Engineering of Graphical User Interfaces for Testing
WCRE '03 Proceedings of the 10th Working Conference on Reverse Engineering
An Overview of the Runtime Verification Tool Java PathExplorer
Formal Methods in System Design
On test suite composition and cost-effective regression testing
ACM Transactions on Software Engineering and Methodology (TOSEM)
Automating regression testing for evolving GUI software: Research Articles
Journal of Software Maintenance and Evolution: Research and Practice - 2003 International Conference on Software Maintenance: The Architectural Evolution of Systems
Studying the Fault-Detection Effectiveness of GUI Test Cases for Rapidly Evolving Software
IEEE Transactions on Software Engineering
An empirical comparison between direct and indirect test result checking approaches
Proceedings of the 3rd international workshop on Software quality assurance
Fourth international workshop on Software quality assurance: in conjunction with the 6th ESEC/FSE joint meeting
Automated gui testing guided by usage profiles
Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering
Covering array sampling of input event sequences for automated gui testing
Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering
Automatically repairing event sequence-based GUI test suites for regression testing
ACM Transactions on Software Engineering and Methodology (TOSEM)
Inter-context control-flow and data-flow test adequacy criteria for nesC applications
Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering
GUI testing using computer vision
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Towards the testing of power-aware software applications for wireless sensor networks
Ada-Europe'07 Proceedings of the 12th international conference on Reliable software technologies
A proposal for automatic testing of GUIs based on annotated use cases
Advances in Software Engineering - Special issue on software test automation
An automated framework for software test oracle
Information and Software Technology
Automated cross-browser compatibility testing
Proceedings of the 33rd International Conference on Software Engineering
Better testing through oracle selection (NIER track)
Proceedings of the 33rd International Conference on Software Engineering
Automated GUI performance testing
Software Quality Control
Accounting for defect characteristics in evaluations of testing techniques
ACM Transactions on Software Engineering and Methodology (TOSEM)
Proceedings of the 34th International Conference on Software Engineering
Recording and replaying navigations on AJAX web sites
ICWE'12 Proceedings of the 12th international conference on Web Engineering
Automated generation of test oracles using a model-driven approach
Information and Software Technology
Graphical user interface (GUI) testing: Systematic mapping and repository
Information and Software Technology
GUITAR: an innovative tool for automated testing of GUI-driven software
Automated Software Engineering
Hi-index | 0.00 |
Test designers widely believe that the overall effectiveness and cost of software testing depends largely on the type and number of test cases executed on the software. This article shows that the test oracle, a mechanism that determines whether a software is executed correctly for a test case, also significantly impacts the fault detection effectiveness and cost of a test case. Graphical user interfaces (GUIs), which have become ubiquitous for interacting with today's software, have created new challenges for test oracle development. Test designers manually “assert” the expected values of specific properties of certain GUI widgets in each test case; during test execution, these assertions are used as test oracles to determine whether the GUI executed correctly. Since a test case for a GUI is a sequence of events, a test designer must decide: (1) what to assert; and (2) how frequently to check an assertion, for example, after each event in the test case or after the entire test case has completed execution. Variations of these two factors significantly impact the fault-detection ability and cost of a GUI test case. A technique to declaratively specify different types of automated GUI test oracles is described. Six instances of test oracles are developed and compared in an experiment on four software systems. The results show that test oracles do affect the fault detection ability of test cases in different and interesting ways: (1) Test cases significantly lose their fault detection ability when using “weak” test oracles; (2) in many cases, invoking a “thorough” oracle at the end of test case execution yields the best cost-benefit ratio; (3) certain test cases detect faults only if the oracle is invoked during a small “window of opportunity” during test execution; and (4) using thorough and frequently-executing test oracles can compensate for not having long test cases.