A Decision-Analytic Stopping Rule for Validation of Commercial Software Systems

• Authors: Tom Chávez
• Venue: IEEE Transactions on Software Engineering
• Year: 2000

The decision about when to release a software product commercially is not a question of when the software has attained some objectively justifiable degree of correctness. It is, rather, a question of whether the software achieves a reasonable balance among engineering objectives, market demand, customer requirements, and marketing directives of the software organization. In this paper, we present a rigorous framework for addressing this important decision. Conjugate distributions from statistical decision theory provide an attractive means of modeling the cost and rate of bugs given information acquired during software testing, as well as prior information provided by software engineers about the fidelity of the software before testing begins. In contrast to methods such as [1] and [15], the stopping analysis presented here yields a computationally simple rule for deciding when to release a commercial software product based on information revealed to engineers during software testing驴complicated numerical procedures are not needed. Our method has the added benefits that it is sequential: It measures explicitly the costs of customer dissatisfaction associated with bugs as well as the costs of declining market position while the testing process continues; and it incorporates a practical framework for cost-criticality assessment that makes sense to professional software developers. A probabilistic model of catastrophic bugs provides another useful way of characterizing and measuring the software's expected performance after commercial release. Taken together, these tools provide a software organization with a clearer basis for making decisions about when to release a commercial software product.