Experimental Models for Validating Technology

• Authors: Marvin V. Zelkowitz;Dolores R. Wallace
• Venue: Computer
• Year: 1998

Experimentation helps determine the effectiveness of proposed theories and methods. But computer science has not developed a concise taxonomy of methods that could be used to demonstrate the validity of new software development techniques. Partly as a result of this, the state of experimentation in software engineering is poor. In this article, the authors describe the four general approaches to experimentation and the important aspects of data collection. They propose a taxonomy for software engineering experimentation that describes 12 different experimental approaches. To underscore the need for experimentation, the authors also analyze 612 papers in software engineering published in 1985, 1990, and 1995. They classified each paper according to the data collection method used to validate the claims in the paper. They found that about a third of the papers had no experimental validation, and many authors used sloppy terminology to describe their validation method. The software engineering community, the authors conclude, can do a better job reporting its results. This, in turn, will help industry select the right new technology to implement.