Using Runtime Analysis to Guide Model Checking of Java Programs
Proceedings of the 7th International SPIN Workshop on SPIN Model Checking and Software Verification
ASE '00 Proceedings of the 15th IEEE international conference on Automated software engineering
RacerX: effective, static detection of race conditions and deadlocks
SOSP '03 Proceedings of the nineteenth ACM symposium on Operating systems principles
Dynamic partial-order reduction for model checking software
Proceedings of the 32nd ACM SIGPLAN-SIGACT symposium on Principles of programming languages
Controlling factors in evaluating path-sensitive error detection techniques
Proceedings of the 14th ACM SIGSOFT international symposium on Foundations of software engineering
Towards a framework and a benchmark for testing tools for multi-threaded programs: Research Articles
Concurrency and Computation: Practice & Experience - Parallel and Distributed Systems: Testing and Debugging (PADTAD)
Parallel Randomized State-Space Search
ICSE '07 Proceedings of the 29th international conference on Software Engineering
Iterative context bounding for systematic testing of multithreaded programs
Proceedings of the 2007 ACM SIGPLAN conference on Programming language design and implementation
Hardness for Explicit State Software Model Checking Benchmarks
SEFM '07 Proceedings of the Fifth IEEE International Conference on Software Engineering and Formal Methods
Race directed random testing of concurrent programs
Proceedings of the 2008 ACM SIGPLAN conference on Programming language design and implementation
Proceedings of the 2008 ACM SIGPLAN conference on Programming language design and implementation
Spin model checker, the: primer and reference manual
Spin model checker, the: primer and reference manual
Guided model checking for programs with polymorphism
Proceedings of the 2009 ACM SIGPLAN workshop on Partial evaluation and program manipulation
A race-detection and flipping algorithm for automated testing of multi-threaded programs
HVC'06 Proceedings of the 2nd international Haifa verification conference on Hardware and software, verification and testing
Generating counter-examples through randomized guided search
Proceedings of the 14th international SPIN conference on Model checking software
BEEM: benchmarks for explicit model checkers
Proceedings of the 14th international SPIN conference on Model checking software
Verifying multi-threaded software using smt-based context-bounded model checking
Proceedings of the 33rd International Conference on Software Engineering
Randomized backtracking in state space traversal
Proceedings of the 18th international SPIN conference on Model checking software
Guided test visualization: Making sense of errors in concurrent programs
ASE '11 Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering
| Hi-index | 0.00 |
In this work we focus on creating a benchmark suite of concurrent programs for various programming languages to evaluate the bug detection capabilities of various tools and techniques. We have compiled a set of Java benchmarks from various sources and our own efforts. For many of the Java examples we have created equivalent C# programs. All the benchmarks are available for download. In our multi-language benchmark suite we compare results from various tools: CalFuzzer, ConTest, CHESS, and Java Pathfinder. In Java Pathfinder we provide extensive results for state-less random walk, randomized depth-first search, and guided search using abstraction refinement. Using data from our study we argue that iterative context-bounding and dynamic partial order reduction are not sufficient to render model checking for testing concurrent programs tractable and secondary techniques such as guidance strategies are required. As part of this work, we have also created a wiki to publish benchmark details and data from various tools on those benchmarks to a broader research community.