Comparing bug finding tools with reviews and tests

Authors:
Stefan Wagner;Jan Jürjens;Claudia Koller;Peter Trischberger
Affiliations:
Institut für Informatik, Technische Universität München, Garching, Germany;Institut für Informatik, Technische Universität München, Garching, Germany;Institut für Informatik, Technische Universität München, Garching, Germany;O2 Germany, Munich, Germany
Venue:
TestCom'05 Proceedings of the 17th IFIP TC6/WG 6.1 international conference on Testing of Communicating Systems
Year:
2005

Citing 15
Cited 15

Software testing techniques (2nd ed.)

Software testing techniques (2nd ed.)
Applied software measurement: assuring productivity and quality

Applied software measurement: assuring productivity and quality
Orthogonal defect classification

Handbook of software reliability engineering
A static analyzer for finding dynamic programming errors

Software—Practice & Experience
Type-based analysis and applications

PASTE '01 Proceedings of the 2001 ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering
The SLAM project: debugging system software via static analysis

POPL '02 Proceedings of the 29th ACM SIGPLAN-SIGACT symposium on Principles of programming languages
Extended static checking for Java

PLDI '02 Proceedings of the ACM SIGPLAN 2002 Conference on Programming language design and implementation
Art of Software Testing

Art of Software Testing
Testing static analysis tools using exploitable buffer overflows from open source code

Proceedings of the 12th ACM SIGSOFT twelfth international symposium on Foundations of software engineering
A Comparison of Bug Finding Tools for Java

ISSRE '04 Proceedings of the 15th International Symposium on Software Reliability Engineering
Finding bugs is easy

ACM SIGPLAN Notices
One evaluation of model-based testing and its automation

Proceedings of the 27th international conference on Software engineering
Check 'n' crash: combining static checking and testing

Proceedings of the 27th international conference on Software engineering
Towards Software Quality Economics for Defect-Detection Techniques

SEW '05 Proceedings of the 29th Annual IEEE/NASA on Software Engineering Workshop
Finding user/kernel pointer bugs with type inference

SSYM'04 Proceedings of the 13th conference on USENIX Security Symposium - Volume 13

Software quality economics for defect-detection techniques using failure prediction

3-WoSQ Proceedings of the third workshop on Software quality
A literature survey of the quality economics of defect-detection techniques

Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
Adaptive Probabilistic Model for Ranking Code-Based Static Analysis Alerts

ICSE COMPANION '07 Companion to the proceedings of the 29th International Conference on Software Engineering
Evaluating static analysis defect warnings on production software

PASTE '07 Proceedings of the 7th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering
DSD-Crasher: A hybrid analysis tool for bug finding

ACM Transactions on Software Engineering and Methodology (TOSEM)
Defect classification and defect types revisited

DEFECTS '08 Proceedings of the 2008 workshop on Defects in large software systems
Securing Java code: heuristics and an evaluation of static analysis tools

Proceedings of the 2008 workshop on Static analysis
EQ-mine: predicting short-term defects for software evolution

FASE'07 Proceedings of the 10th international conference on Fundamental approaches to software engineering
Making defect-finding tools work for you

Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2
Continual monitoring of code quality

Proceedings of the 4th India Software Engineering Conference
Exploring the relationship betweenweb application development tools and security

WebApps'11 Proceedings of the 2nd USENIX conference on Web application development
A systematic mapping study on the combination of static and dynamic quality assurance techniques

Information and Software Technology
Residual investigation: predictive and precise bug detection

Proceedings of the 2012 International Symposium on Software Testing and Analysis
An evaluation of the Google Chrome extension security architecture

Security'12 Proceedings of the 21st USENIX conference on Security symposium
Static correspondence and correlation between field defects and warnings reported by a bug finding tool

Software Quality Control

Quantified Score

Hi-index	0.00

Visualization

Abstract

Bug finding tools can find defects in software source code using an automated static analysis. This automation may be able to reduce the time spent for other testing and review activities. For this we need to have a clear understanding of how the defects found by bug finding tools relate to the defects found by other techniques. This paper describes a case study using several projects mainly from an industrial environment that were used to analyse the interrelationships. The main finding is that the bug finding tools predominantly find different defects than testing but a subset of defects found by reviews. However, the types that can be detected are analysed more thoroughly. Therefore, a combination is most advisable if the high number of false positives of the tools can be tolerated.