Applying design of experiments to software testing: experience report
ICSE '97 Proceedings of the 19th international conference on Software engineering
The AETG System: An Approach to Testing Based on Combinatorial Design
IEEE Transactions on Software Engineering
Symbolic execution and program testing
Communications of the ACM
CIL: Intermediate Language and Tools for Analysis and Transformation of C Programs
CC '02 Proceedings of the 11th International Conference on Compiler Construction
An Investigation of the Applicability of Design of Experiments to Software Testing
SEW '02 Proceedings of the 27th Annual NASA Goddard Software Engineering Workshop (SEW-27'02)
DART: directed automated random testing
Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation
EXE: automatically generating inputs of death
Proceedings of the 13th ACM conference on Computer and communications security
Skoll: A Process and Infrastructure for Distributed Continuous Quality Assurance
IEEE Transactions on Software Engineering
A decision procedure for bit-vectors and arrays
CAV'07 Proceedings of the 19th international conference on Computer aided verification
KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs
OSDI'08 Proceedings of the 8th USENIX conference on Operating systems design and implementation
Mixing type checking and symbolic execution
PLDI '10 Proceedings of the 2010 ACM SIGPLAN conference on Programming language design and implementation
Static extraction of program configuration options
Proceedings of the 33rd International Conference on Software Engineering
Pairwise feature-interaction testing for SPLs: potentials and limitations
Proceedings of the 15th International Software Product Line Conference, Volume 2
SAS'11 Proceedings of the 18th international conference on Static analysis
An empirical study on configuration errors in commercial and open source systems
SOSP '11 Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles
Specifying and verifying the correctness of dynamic software updates
VSTTE'12 Proceedings of the 4th international conference on Verified Software: theories, tools, experiments
Integration testing of software product lines using compositional symbolic execution
FASE'12 Proceedings of the 15th international conference on Fundamental Approaches to Software Engineering
iTree: efficiently discovering high-coverage configurations using interaction trees
Proceedings of the 34th International Conference on Software Engineering
A variability-aware module system
Proceedings of the ACM international conference on Object oriented programming systems languages and applications
An analysis of unit tests of a flight software product line
Science of Computer Programming
An analysis of unit tests of a flight software product line
Science of Computer Programming
| Hi-index | 0.00 |
Many modern software systems are designed to be highly configurable, which increases flexibility but can make programs hard to test, analyze, and understand. We present an initial empirical study of how configuration options affect program behavior. We conjecture that, at certain levels of abstraction, configuration spaces are far smaller than the worst case, in which every configuration is distinct. We evaluated our conjecture by studying three configurable software systems: vsftpd, ngIRCd, and grep. We used symbolic evaluation to discover how the settings of run-time configuration options affect line, basic block, edge, and condition coverage for our subjects under a given test suite. Our results strongly suggest that for these subject programs, test suites, and configuration options, when abstracted in terms of the four coverage criteria above, configuration spaces are in fact much smaller than combinatorics would suggest and are effectively the composition of many small, self-contained groupings of options.