State coverage: software validation metrics beyond code coverage

Authors:
Dries Vanoverberghe;Jonathan de Halleux;Nikolai Tillmann;Frank Piessens
Affiliations:
Katholieke Universiteit Leuven, Leuven, Belgium;Microsoft Research, Redmond, WA;Microsoft Research, Redmond, WA;Katholieke Universiteit Leuven, Leuven, Belgium
Venue:
SOFSEM'12 Proceedings of the 38th international conference on Current Trends in Theory and Practice of Computer Science
Year:
2012

Citing 19
Cited 0

Selecting Software Test Data Using Data Flow Information

IEEE Transactions on Software Engineering
A Practical Approach to Programming With Assertions

IEEE Transactions on Software Engineering
Structural specification-based testing with ADL

ISSTA '96 Proceedings of the 1996 ACM SIGSOFT international symposium on Software testing and analysis
Software unit test coverage and adequacy

ACM Computing Surveys (CSUR)
Symbolic execution and program testing

Communications of the ACM
A Metrics Suite for Object Oriented Design

IEEE Transactions on Software Engineering
Assertions: A Personal Perspective

IEEE Annals of the History of Computing
Assertions in programming languages

ACM SIGPLAN Notices
Assessing the Relationship between Software Assertions and Faults: An Empirical Investigation

ISSRE '06 Proceedings of the 17th International Symposium on Software Reliability Engineering
State coverage: a structural test adequacy criterion for behavior checking

Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
Hints on Test Data Selection: Help for the Practicing Programmer

Computer
A Complexity Measure

IEEE Transactions on Software Engineering
UnitPlus: assisting developer testing in Eclipse

Proceedings of the 2007 OOPSLA workshop on eclipse technology eXchange
Review: A systematic review of software fault prediction studies

Expert Systems with Applications: An International Journal
The Art of Unit Testing: With Examples in .Net

The Art of Unit Testing: With Examples in .Net
Embedded contract languages

Proceedings of the 2010 ACM Symposium on Applied Computing
Pex: white box test generation for .NET

TAP'08 Proceedings of the 2nd international conference on Tests and proofs
Specification test coverage adequacy criteria = specification test generation inadequacy criteria

HASE'04 Proceedings of the Eighth IEEE international conference on High assurance systems engineering
Language-based information-flow security

IEEE Journal on Selected Areas in Communications

Quantified Score

Hi-index	0.00

Visualization

Abstract

Currently, testing is still the most important approach to reduce the amount of software defects. Software quality metrics help to prioritize where additional testing is necessary by measuring the quality of the code. Most approaches to estimate whether some unit of code is sufficiently tested are based on code coverage, which measures what code fragments are exercised by the test suite. Unfortunately, code coverage does not measure to what extent the test suite checks the intended functionality. We propose state coverage , a metric that measures the ratio of state updates that are read by assertions with respect to the total number of state updates, and we present efficient algorithms to measure state coverage. Like code coverage, state coverage is simple to understand and we show that it is effective to measure and easy to aggregate. During a preliminary evaluation on several open-source libraries, state coverage helped to identify multiple unchecked properties and detect several bugs.