Architecture-based testing using goals and plans
Proceedings of the ISSTA 2006 workshop on Role of software architecture for testing and analysis
Software testing: perception on exploration and ad-libbing
MACMESE'09 Proceedings of the 11th WSEAS international conference on Mathematical and computational methods in science and engineering
| Hi-index | 0.01 |
The "oracle problem" is a well-known challenge for software testing. Without some means of automatically computing the correct answer for test cases, testers must instead compute the results by hand, or use a previous version of the software. In this paper, we investigate the feasibility of revealing software faults by augmenting the code with complete, specification-based assertions. Our evaluation method is to (1) develop a formal specification, (2) translate this specification into assertions, (3) inject or identify existing faults, and (4) for each version of the assertion-enhanced system containing a fault, execute it using a set of test inputs and check for assertion violations. Our goal is to determine whether specification-based assertions are a viable method of revealing faults, and to begin to assess the extent to which their cost-effectiveness can be improved. Our evaluation is based on two case studies involving real-world software systems. Our results indicate that specification-based assertions can effectively reveal faults, as long as they adversely affect the program state. We describe techniques that we used for translating high-level specifications into code-level assertions. We also discuss the costs associated with the approach, and potential techniques for reducing these costs.