An empirical study of the reliability of UNIX utilities
Communications of the ACM
Automated test data generation for programs with procedures
ISSTA '96 Proceedings of the 1996 ACM SIGSOFT international symposium on Software testing and analysis
Journal of the ACM (JACM)
QuickCheck: a lightweight tool for random testing of Haskell programs
ICFP '00 Proceedings of the fifth ACM SIGPLAN international conference on Functional programming
Korat: automated testing based on Java predicates
ISSTA '02 Proceedings of the 2002 ACM SIGSOFT international symposium on Software testing and analysis
The nofib Benchmark Suite of Haskell Programs
Proceedings of the 1992 Glasgow Workshop on Functional Programming
Test input generation with java PathFinder
ISSTA '04 Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis
The Implementation of Functional Programming Languages (Prentice-Hall International Series in Computer Science)
Using model checking to find serious file system errors
OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Systematic generation of glass-box test cases for functional logic programs
Proceedings of the 9th ACM SIGPLAN international conference on Principles and practice of declarative programming
Haskell '07 Proceedings of the ACM SIGPLAN workshop on Haskell workshop
Smallcheck and lazy smallcheck: automatic exhaustive testing for small values
Proceedings of the first ACM SIGPLAN symposium on Haskell
Not all patterns, but enough: an automatic verifier for partial but sufficient pattern matching
Proceedings of the first ACM SIGPLAN symposium on Haskell
Finding the needle: stack traces for GHC
Proceedings of the 2nd ACM SIGPLAN symposium on Haskell
RWset: attacking path explosion in constraint-based test generation
TACAS'08/ETAPS'08 Proceedings of the Theory and practice of software, 14th international conference on Tools and algorithms for the construction and analysis of systems
KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs
OSDI'08 Proceedings of the 8th USENIX conference on Operating systems design and implementation
Eclat: automatic generation and classification of test inputs
ECOOP'05 Proceedings of the 19th European conference on Object-Oriented Programming
Hi-index | 0.00 |
This paper presents a new lightweight technique for automatically generating high coverage test suites for Haskell library code. Our approach combines four main features to increase test coverage: (1) automatically inferring the constructors and functions needed to generate test data; (2) using needed narrowing to take advantage of Haskell's lazy evaluation semantics; (3) inspecting elements inside returned data structures through the use of case statements, and (4) efficiently handling polymorphism by lazily instantiating all possible instances. We have implemented this technique in IRULAN, a fully automatic tool for systematic black-box unit testing of Haskell library code. We have designed IRULAN to generate high coverage test suites and detect common programming errors in the process. We have applied IRULAN to over 50 programs from the spectral and real suites of the nofib benchmark and show that it can effectively generate high-coverage test suites---exhibiting 70.83% coverage for spectral and 59.78% coverage for real---and find errors in these programs. Our techniques are general enough to be useful for several other types of testing, and we also discuss our experience of using IRULAN for property and regression testing.