Evaluation of Regressive Methods for Automated Generation of Test Trajectories

• Authors: Brian J. Taylor;Bojan Cukic
• Venue: ISSRE '00 Proceedings of the 11th International Symposium on Software Reliability Engineering
• Year: 2000

Automated generation of test cases is a prerequisite for fast testing. Whereas the research has addressed the creation of individual test points, test trajectory generation has attracted limited. In simple terms, a test trajectory is defined as a series of data points, with each (possibly multidimensional) point relying upon the value(s) of previous point(s). Software systems that use data trajectories as inputs include closed-loop process controllers. For these systems, software testers can handcraft test trajectories, use input trajectories from older versions of the system or, perhaps, collect test data in a high fidelity system simulator. While these are valid approaches, they are expensive and time-consuming, especially if the assessment goals require substantial number of tests.In this paper, we propose a framework for expanding a small, conventionally developed set of test trajectories into a large set suitable, for example, for system safety assurance. In the core of this framework is statistical regression analysis. The regression analysis builds a relationship between controllable independent variables and closely correlated dependent variables, which represent test trajectories. By perturbing the independent variables, new test trajectories can be generated automatically. Automated test trajectory generation has been applied in the safety assessment of a fault tolerant flight control system. We compare the performance of simple linear regression, multiple linear regression, and autoregressive techniques. The performance metrics include the speed of test generation and the percentage of 驴acceptable驴 trajectories, measured by the domain specific reasonableness checks.