Automated testing and debugging of SAT and QBF solvers

Authors:
Robert Brummayer;Florian Lonsing;Armin Biere
Affiliations:
Institute for Formal Models and Verification, Johannes Kepler University, Linz, Austria;Institute for Formal Models and Verification, Johannes Kepler University, Linz, Austria;Institute for Formal Models and Verification, Johannes Kepler University, Linz, Austria
Venue:
SAT'10 Proceedings of the 13th international conference on Theory and Applications of Satisfiability Testing
Year:
2010

Citing 30
Cited 5

Resolution for quantified Boolean formulas

Information and Computation
Improvements to propositional satisfiability search algorithms

Improvements to propositional satisfiability search algorithms
Beyond NP: the QSAT phase transition

AAAI '99/IAAI '99 Proceedings of the sixteenth national conference on Artificial intelligence and the eleventh Innovative applications of artificial intelligence conference innovative applications of artificial intelligence
QuickCheck: a lightweight tool for random testing of Haskell programs

ICFP '00 Proceedings of the fifth ACM SIGPLAN international conference on Functional programming
A machine program for theorem-proving

Communications of the ACM
Simplifying and Isolating Failure-Inducing Input

IEEE Transactions on Software Engineering
An Algorithm to Evaluate Quantified Boolean Formulae and Its Experimental Evaluation

Journal of Automated Reasoning
Lemma and Model Caching in Decision Procedures for Quantified Boolean Formulas

TABLEAUX '02 Proceedings of the International Conference on Automated Reasoning with Analytic Tableaux and Related Methods
Towards a Symmetric Treatment of Satisfaction and Conflicts in Quantified Boolean Formula Evaluation

CP '02 Proceedings of the 8th International Conference on Principles and Practice of Constraint Programming
Validating SAT Solvers Using an Independent Resolution-Based Checker: Practical Implementations and Other Applications

DATE '03 Proceedings of the conference on Design, Automation and Test in Europe - Volume 1
Why Programs Fail: A Guide to Systematic Debugging

Why Programs Fail: A Guide to Systematic Debugging
Validating the result of a Quantified Boolean Formula (QBF) solver: theory and practice

Proceedings of the 2005 Asia and South Pacific Design Automation Conference
HDD: hierarchical delta debugging

Proceedings of the 28th international conference on Software engineering
Multithreaded SAT Solving

ASP-DAC '07 Proceedings of the 2007 Asia and South Pacific Design Automation Conference
Fuzzing: Brute Force Vulnerability Discovery

Fuzzing: Brute Force Vulnerability Discovery
Fuzzing for Software Security Testing and Quality Assurance

Fuzzing for Software Security Testing and Quality Assurance
The Barcelogic SMT Solver

CAV '08 Proceedings of the 20th international conference on Computer Aided Verification
Handbook of Satisfiability: Volume 185 Frontiers in Artificial Intelligence and Applications

Handbook of Satisfiability: Volume 185 Frontiers in Artificial Intelligence and Applications
VARSAT: Integrating Novel Probabilistic Inference Techniques with DPLL Search

SAT '09 Proceedings of the 12th International Conference on Theory and Applications of Satisfiability Testing
The Conflict-Driven Answer Set Solver clasp: Progress Report

LPNMR '09 Proceedings of the 10th International Conference on Logic Programming and Nonmonotonic Reasoning
Generating hard SAT/CSP instances using expander graphs

AAAI'08 Proceedings of the 23rd national conference on Artificial intelligence - Volume 3
A model for generating random quantified boolean formulas

IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
A simple model to generate hard satisfiable instances

IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Evaluating and certifying QBFs: A comparison of state-of-the-art tools

AI Communications
Fuzzing and delta-debugging SMT solvers

Proceedings of the 7th International Workshop on Satisfiability Modulo Theories
A first step towards a unified proof checker for QBF

SAT'07 Proceedings of the 10th international conference on Theory and applications of satisfiability testing
New results on the phase transition for random quantified Boolean formulas

SAT'08 Proceedings of the 11th international conference on Theory and applications of satisfiability testing
Exploiting a theory of phase transitions in three-satisfiability problems

AAAI'96 Proceedings of the thirteenth national conference on Artificial intelligence - Volume 1
Resolve and expand

SAT'04 Proceedings of the 7th international conference on Theory and Applications of Satisfiability Testing
sKizzo: a suite to evaluate and certify QBFs

CADE' 20 Proceedings of the 20th international conference on Automated Deduction

Fuzzing the out-of-memory killer on embedded Linux: an adaptive random approach

Proceedings of the 2011 ACM Symposium on Applied Computing
versat: a verified modern SAT solver

VMCAI'12 Proceedings of the 13th international conference on Verification, Model Checking, and Abstract Interpretation
Creating industrial-like SAT instances by clustering and reconstruction

SAT'12 Proceedings of the 15th international conference on Theory and Applications of Satisfiability Testing
A framework for the specification of random SAT and QSAT formulas

TAP'12 Proceedings of the 6th international conference on Tests and Proofs
Verifying refutations with extended resolution

CADE'13 Proceedings of the 24th international conference on Automated Deduction

Quantified Score

Hi-index	0.00

Visualization

Abstract

Robustness and correctness are essential criteria for SAT and QBF solvers. We develop automated testing and debugging techniques designed and optimized for SAT and QBF solver development. Our fuzz testing techniques are able to find critical solver defects that lead to crashes, invalid satisfying assignments and incorrect satisfiability results. Moreover, we show that sequential and concurrent delta debugging techniques are highly effective in minimizing failure-inducing inputs.