Ranking programs using black box testing

Authors:
Koen Claessen;John Hughes;Michał Pałka;Nick Smallbone;Hans Svensson
Affiliations:
Chalmers University of Technology, Gothenburg, Sweden;Chalmers University of Technology and Quviq AB, Gothenburg, Sweden;Chalmers University of Technology, Gothenburg, Sweden;Chalmers University of Technology, Gothenburg, Sweden;Chalmers University of Technology and Quviq AB, Gothenburg, Sweden
Venue:
Proceedings of the 5th Workshop on Automation of Software Test
Year:
2010

Citing 14
Cited 1

Experimentation in software engineering

IEEE Transactions on Software Engineering
Comparing the Effectiveness of Software Testing Strategies

IEEE Transactions on Software Engineering
Experimentation in software engineering: an introduction

Experimentation in software engineering: an introduction
QuickCheck: a lightweight tool for random testing of Haskell programs

ICFP '00 Proceedings of the fifth ACM SIGPLAN international conference on Functional programming
A controlled experiment in program testing and code walkthroughs/inspections

Communications of the ACM
Isolating cause-effect chains from computer programs

Proceedings of the 10th ACM SIGSOFT symposium on Foundations of software engineering
Is Proof More Cost-Effective Than Testing?

IEEE Transactions on Software Engineering
Preliminary guidelines for empirical research in software engineering

IEEE Transactions on Software Engineering
Testing for software reliability

Proceedings of the international conference on Reliable software
On the Effectiveness of the Test-First Approach to Programming

IEEE Transactions on Software Engineering
Parameterized unit tests

Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering
Testing telecoms software with quviq QuickCheck

Proceedings of the 2006 ACM SIGPLAN workshop on Erlang
Software Testing Research: Achievements, Challenges, Dreams

FOSE '07 2007 Future of Software Engineering
Pex: white box test generation for .NET

TAP'08 Proceedings of the 2nd international conference on Tests and proofs

Software testing with QuickCheck

CEFP'09 Proceedings of the Third summer school conference on Central European functional programming school

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present an unbiased method for measuring the relative quality of different solutions to a programming problem. Our method is based on identifying possible bugs from program behaviour through black-box testing. The main motivation for such a method is its use in experimental evaluation of software development methods. We report on the use of our method in a small-scale such experiment, which was aimed at evaluating the effectiveness of property-based testing vs. unit testing in software development.