Trace Analysis for Conformance and Arbitration Testing
IEEE Transactions on Software Engineering
Specification-based test oracles for reactive systems
ICSE '92 Proceedings of the 14th international conference on Software engineering
Oracles for checking temporal properties of concurrent systems
SIGSOFT '94 Proceedings of the 2nd ACM SIGSOFT symposium on Foundations of software engineering
Generating oracles from your favorite temporal logic specifications
SIGSOFT '96 Proceedings of the 4th ACM SIGSOFT symposium on Foundations of software engineering
Using Test Oracles Generated from Program Documentation
IEEE Transactions on Software Engineering
Testing generic Ada packages with APE
Proceedings of the 1998 annual ACM SIGAda international conference on Ada
Specifying Concurrent Systems with TSL
IEEE Software
Compilation of Z Specifications into C for Automatic Test Result Evaluation
ZUM '95 Proceedings of the 9th International Conference of Z Usres on The Z Formal Specification Notation
Improving Software Tests Using Z Specifications
ZUM '95 Proceedings of the 9th International Conference of Z Usres on The Z Formal Specification Notation
Translating Object-Z Specifications to Passive Test Oracles
ICFEM '98 Proceedings of the Second IEEE International Conference on Formal Engineering Methods
Automated replay and failure detection for web applications
Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering
Hi-index | 0.01 |
The problem of verifying the correctness of test executions is well known: while manual verification is time-consuming and error-prone, developing an oracle to automatically verify test executions can be as costly as implementing the original program. This is especially true for concurrent programs, due to their non-determinism and complexity. In this paper, we present a method that uses partial specifications to systematically derive oracles for concurrent programs. We illustrate the method by deriving an Ada task that monitors the execution of a concurrent Ada program and describe a prototype tool that partially automates the derivation process. We present the results of a study that shows the derived oracles are surprisingly effective at error detection. The study also shows that manual verification is an inaccurate means of failure detection, that large test case sets must be used to ensure adequate testing coverage, and that test cases must be run many times to cover for variations in run-time behaviour.