Generation of TLM testbenches using mutation testing

• Authors: Marcelo Sousa;Alper Sen
• Venue: Proceedings of the eighth IEEE/ACM/IFIP international conference on Hardware/software codesign and system synthesis
• Year: 2012

Testbench development is a major component of simulation based verification, which is the de-facto verification technique used in the industry. Verification of a TLM design is not complete without a measure of the effectiveness of its testbenches. We devise a coverage driven testbench generation technique where the coverage metric uses that of a fault insertion based approach, namely mutation testing. Mutation testing is a commonly used software testing technique to measure the quality of testbenches. In mutation testing, the goal is to insert mutations (syntactic changes) into the program and check whether the impact of these mutations on the program can be detected by the available testbenches. If the testbenches cannot detect the impact of these inserted mutations then one can potentially add new testbenches to detect these changes. In this work, we automate the process of mutation testing based testbench generation exploiting the properties of concurrent TLM designs. Our framework is novel in that it uses the byte code representation of SystemC TLM models using Low Level Virtual Machine (LLVM) framework. Furthermore, these testbenches can detect concurrency related defects such as deadlocks or race conditions. We perform experiments on TLM models to demonstrate the effectiveness of our test bench generation approach.