Orthogonal array application for optimal combination of software defect detection techniques choices

• Authors: Ljubomir Lazic;Nikos Mastorakis
• Venue: WSEAS Transactions on Computers
• Year: 2008

In this paper, we consider a problem that arises in black box testing: generating small test suites (i.e., sets of test cases) where the combinations that have to be covered are specified by input-output parameter relationships of a software system. That is, we only consider combinations of input parameters that affect an output parameter, and we do not assume that the input parameters have the same number of values. To solve this problem, we propose interaction testing, particularly an Orthogonal Array Testing Strategy (OATS) as a systematic, statistical way of testing pair-wise interactions. In software testing process (STP), it provides a natural mechanism for testing systems to be deployed on a variety of hardware and software configurations. The combinatorial approach to software testing uses models to generate a minimal number of test inputs so that selected combinations of input values are covered. The most common coverage criteria are two-way or pairwise coverage of value combinations, though for higher confidence three-way or higher coverage may be required. This paper presents some examples of software-system test requirements and corresponding models for applying the combinatorial approach to those test requirements. The method bridges contributions from mathematics, design of experiments, software test, and algorithms for application to usability testing. Also, this study presents a brief overview of the response surface methods (RSM) for computer experiments available in the literature. The Bayesian approach and orthogonal arrays constructed for computer experiments (OACE) were briefly discussed. An example, of a novel OACE application, to STP optimization study was also given. In this case study, an orthogonal array for computer experiments was utilized to build a second order response surface model. Gradient-based optimization algorithms could not be utilized in this case study since the design variables were discrete valued. Using OACE novel approach, optimum combination of software defect detection techniques choices for every software development phase that maximize all over Defect Detection Effectiveness of STP were determined.