NPATH: a measure of execution path complexity and its applications
Communications of the ACM
Managing the software process
Safety-critical software: a vignette
Software Engineering Journal
Science and Substance: A Challenge to Software Engineers
IEEE Software
Experience with Formal Methods in Critical Systems
IEEE Software
Observations on industrial practice using formal methods
ICSE '93 Proceedings of the 15th international conference on Software Engineering
Safer C: Developing Software for in High-Integrity and Safety-Critical Systems
Safer C: Developing Software for in High-Integrity and Safety-Critical Systems
CICS Project Report: Experiences and Results from the use of Z in IBM
VDM '91 Proceedings of the 4th International Symposium of VDM Europe on Formal Software Development-Volume I: Conference Contributions - Volume I
From formal models to formally based methods: an industrial experience
ACM Transactions on Software Engineering and Methodology (TOSEM)
Validation of HOL Proofs by Proof Checking
Formal Methods in System Design
A framework architecture for supervision and control systems
ACM Computing Surveys (CSUR)
Striving for mathematical thinking
Working group reports from ITiCSE on Innovation and technology in computer science education
Striving for mathematical thinking
ACM SIGCSE Bulletin
The T Experiments: Errors In Scientific Software
IEEE Computational Science & Engineering
N-Version Design Versus One Good Version
IEEE Software
Software Quality's Eight Greatest Myths
IEEE Software
Is Proof More Cost-Effective Than Testing?
IEEE Transactions on Software Engineering
Quantitative Analysis of Faults and Failures in a Complex Software System
IEEE Transactions on Software Engineering
From Complex Specifications to a Working Prototype. A Protocol Engineering Case Study
FME '01 Proceedings of the International Symposium of Formal Methods Europe on Formal Methods for Increasing Software Productivity
The Value of Verification: Positive Experience of Industrial Proof
FM '99 Proceedings of the Wold Congress on Formal Methods in the Development of Computing Systems-Volume II
Journal of Systems and Software
Formal modeling in a commercial setting: a case study
Journal of Systems and Software
What Does Industry Need From Formal Specification Techniques?
WIFT '98 Proceedings of the Second IEEE Workshop on Industrial Strength Formal Specification Techniques
Studying Software Engineers: Data Collection Techniques for Software Field Studies
Empirical Software Engineering
Quality, cleanroom and formal methods
3-WoSQ Proceedings of the third workshop on Software quality
An Experimental Investigation of Formality in UML-Based Development
IEEE Transactions on Software Engineering
Language subsetting in an industrial context: A comparison of MISRA C 1998 and MISRA C 2004
Information and Software Technology
A Replicated Quantitative Analysis of Fault Distributions in Complex Software Systems
IEEE Transactions on Software Engineering
Melange: creating a "functional" internet
Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007
A direct path to dependable software
Communications of the ACM - A Direct Path to Dependable Software
Exploration of SWRL Rule Bases through Visualization, Paraphrasing, and Categorization of Rules
RuleML '09 Proceedings of the 2009 International Symposium on Rule Interchange and Applications
An automated framework for software test oracle
Information and Software Technology
Artificial neural networks as multi-networks automated test oracle
Automated Software Engineering
Redevelopment of an industrial case study using event-B and Rodin
FACS-FMI'07 Proceedings of the 2007th internatioanal conference on Formal Methods in Industry
Hi-index | 4.10 |
Formal methods promise much, but can they deliver? In this project, results are inconclusive, but careful data gathering and analysis helped establish influences on product quality. Practitioners and researchers continue to seek methods and tools for improving software development processes and products. Candidate technologies promise increased productivity, better quality, lower cost, or enhanced customer satisfaction. But we must test these methods and tools empirically and rigorously to determine any significant, quantifiable improvement. We tend to consider evaluation only after using the technology, which makes careful, quantitative analysis difficult if not impossible. However, when an evaluation is designed as part of overall project planning, and then carried out as software development progresses, the result can be a rich record of a tool's or technique's effectiveness. In this study, we investigated the effects of using formal methods to develop an air-traffic-control information system. Because we are studying one project in isolation, we cannot draw conclusions about the suitability of formal methods for all projects. As we describe in the sidebar "Can Formal Methods Always Deliver?" the jury is still out on when and whether formal methods improve products. Nevertheless, the lessons we learned are instructive, not only in showing how formal methods influenced code quality on this project, but also in highlighting the limitations of retrospective studies and their use in planning follow-up investigations. We describe what we did, as well as what we could have done had the study been carried out as the software system was being developed and tested. We also show how this preliminary investigation helps to suggest hypotheses for further studies. Thus, the lessons we learned can be applied not only to gauge the effects of formal methods but also in planning similar studies of other techniques and tools. The procedure we used was not predetermined; the results of one analysis step largely determined where we went next. Indeed, research often involves following one trail and then another, uncovering relationships and unearthing facts, until the picture begins to make sense. However, we did learn many specific lessons, which we hope will enrich future investigations.