A framework for the evaluation of specification miners based on finite state machines

Authors:
Michael Pradel;Philipp Bichsel;Thomas R. Gross
Affiliations:
Computer Science Department, ETH Zurich, Switzerland;Computer Science Department, ETH Zurich, Switzerland;Computer Science Department, ETH Zurich, Switzerland
Venue:
ICSM '10 Proceedings of the 2010 IEEE International Conference on Software Maintenance
Year:
2010

Citing 0
Cited 8

Detecting anomalies in the order of equally-typed method arguments

Proceedings of the 2011 International Symposium on Software Testing and Analysis
An empirical study of object protocols in the wild

Proceedings of the 25th European conference on Object-oriented programming
A case study on business process recovery using an e-government system

Software—Practice & Experience
Learning extended FSA from software: An empirical assessment

Journal of Systems and Software
Leveraging test generation and specification mining for automated bug detection without false positives

Proceedings of the 34th International Conference on Software Engineering
Statically checking API protocol conformance with mined multi-object specifications

Proceedings of the 34th International Conference on Software Engineering
Automated Comparison of State-Based Software Models in Terms of Their Language and Structure

ACM Transactions on Software Engineering and Methodology (TOSEM)
Unifying FSM-inference algorithms through declarative specification

Proceedings of the 2013 International Conference on Software Engineering

Quantified Score

Hi-index	0.00

Visualization

Abstract

Software maintenance tasks, such as testing and program understanding, can benefit from formal specifications that describe how a program should use an API. Recently, there has been increasing interest in specification miners that automatically extract finite state specifications of method ordering constraints from existing software. However, comparing different mining approaches is difficult, because no common ground to evaluate the effectiveness of specification miners has been established yet. We present a framework for evaluating to which extent specification miners find valid finite state descriptions of API usage constraints. The framework helps in creating reference specifications and includes metrics to compare mined specifications to the reference specifications. The metrics are tailored for evaluating specification miners and account for imprecision and incompleteness in mined specifications. We use the framework to compare the effectiveness of three mining approaches and to show their respective benefits.