Automatic system testing of programs without test oracles
Proceedings of the eighteenth international symposium on Software testing and analysis
Automated metamorphic testing on the analyses of feature models
Information and Software Technology
Testing and validating machine learning classifiers by metamorphic testing
Journal of Systems and Software
Testing a binary space partitioning algorithm with metamorphic testing
Proceedings of the 2011 ACM Symposium on Applied Computing
Metamorphic testing of a Monte Carlo modeling program
Proceedings of the 6th International Workshop on Automation of Software Test
Hi-index | 0.00 |
It is challenging to test applications and functions for which the correct output for arbitrary input cannot be known in advance, e.g. some computational science or machine learning applications. In the absence of a test oracle, one approach to testing these applications is to use metamorphic testing: existing test case input is modified to produce new test cases in such a manner that, when given the new input, the application should produce an output that can be easily be computed based on the original output. That is, if input x produces output f(x), then we create input x' such that we can predict f(x') based on f(x); if the application or function does not produce the expected output, then a defect must exist, and either f(x) or f(x') (or both) is wrong. By using metamorphic testing, we are able to provide built-in "pseudo-oracles" for these so-called "nontestable programs" that have no test oracles.In this paper, we describe an approach in which a function's metamorphic properties are specified using an extension to the Java Modeling Language (JML), a behavioral interface specification language that is used to support the "design by contract" paradigm in Java applications. Our implementation, called Corduroy, pre-processes these specifications and generates test code that can be executed using JML runtime assertion checking, for ensuring that the specifications hold during program execution. In addition topresenting our approach and implementation, we also describe our findings from case studies in which we apply our technique to applications without test oracles.