Residual test coverage monitoring
Proceedings of the 21st international conference on Software engineering
Multivariate visualization in observation-based testing
Proceedings of the 22nd international conference on Software engineering
Extracting usability information from user interface events
ACM Computing Surveys (CSUR)
Finding failures by cluster analysis of execution profiles
ICSE '01 Proceedings of the 23rd International Conference on Software Engineering
Pursuing failure: the distribution of program failures in a profile space
Proceedings of the 8th European software engineering conference held jointly with 9th ACM SIGSOFT international symposium on Foundations of software engineering
Machine Learning
Monitoring deployed software using software tomography
Proceedings of the 2002 ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering
Visualization of program-execution data for deployed software
Proceedings of the 2003 ACM symposium on Software visualization
Automated support for classifying software failure reports
Proceedings of the 25th International Conference on Software Engineering
Bug isolation via remote program sampling
PLDI '03 Proceedings of the ACM SIGPLAN 2003 conference on Programming language design and implementation
Leveraging field data for impact analysis and regression testing
Proceedings of the 9th European software engineering conference held jointly with 11th ACM SIGSOFT international symposium on Foundations of software engineering
Pattern Classification (2nd Edition)
Pattern Classification (2nd Edition)
Kernel Methods for Pattern Analysis
Kernel Methods for Pattern Analysis
Finding Latent Code Errors via Machine Learning over Program Executions
Proceedings of the 26th International Conference on Software Engineering
Covering arrays for efficient fault characterization in complex configuration spaces
ISSTA '04 Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis
Active learning for automatic classification of software behavior
ISSTA '04 Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis
Tree-Based Methods for Classifying Software Failures
ISSRE '04 Proceedings of the 15th International Symposium on Software Reliability Engineering
Scalable statistical bug isolation
Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation
Selective capture and replay of program executions
WODA '05 Proceedings of the third international workshop on Dynamic analysis
A historical perspective on runtime assertion checking in software development
ACM SIGSOFT Software Engineering Notes
Failure proximity: a fault localization-based approach
Proceedings of the 14th ACM SIGSOFT international symposium on Foundations of software engineering
On-line anomaly detection of deployed software: a statistical machine learning approach
Proceedings of the 3rd international workshop on Software quality assurance
Techniques for Classifying Executions of Deployed Software to Support Software Engineering Tasks
IEEE Transactions on Software Engineering
An iterative, multi-level, and scalable approach to comparing execution traces
Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
An iterative, multi-level, and scalable approach to comparing execution traces
The 6th Joint Meeting on European software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering: companion papers
Context-aware statistical debugging: from bug predictors to faulty control flow paths
Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering
Predicting buggy changes inside an integrated development environment
Proceedings of the 2007 OOPSLA workshop on eclipse technology eXchange
The probabilistic program dependence graph and its application to fault diagnosis
ISSTA '08 Proceedings of the 2008 international symposium on Software testing and analysis
Profile-guided program simplification for effective testing and analysis
Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering
SIFT: a scalable iterative-unfolding technique for filtering execution traces
CASCON '08 Proceedings of the 2008 conference of the center for advanced studies on collaborative research: meeting of minds
Combining hardware and software instrumentation to classify program executions
Proceedings of the eighteenth ACM SIGSOFT international symposium on Foundations of software engineering
Monitoring, analysis, and testing of deployed software
Proceedings of the FSE/SDP workshop on Future of software engineering research
Using entropy measures for comparison of software traces
Information Sciences: an International Journal
Automatically finding performance problems with feedback-directed learning software testing
Proceedings of the 34th International Conference on Software Engineering
Is this a bug or an obsolete test?
ECOOP'13 Proceedings of the 27th European conference on Object-Oriented Programming
Hi-index | 0.00 |
There is an increasing interest in techniques that support measurement and analysis of fielded software systems. One of the main goals of these techniques is to better understand how software actually behaves in the field. In particular, many of these techniques require a way to distinguish, in the field, failing from passing executions. So far, researchers and practitioners have only partially addressed this problem: they have simply assumed that program failure status is either obvious (i.e., the program crashes) or provided by an external source (e.g., the users). In this paper, we propose a technique for automatically classifying execution data, collected in the field, as coming from either passing or failing program runs. (Failing program runs are executions that terminate with a failure, such as a wrong outcome.) We use statistical learning algorithms to build the classification models. Our approach builds the models by analyzing executions performed in a controlled environment (e.g., test cases run in-house) and then uses the models to predict whether execution data produced by a fielded instance were generated by a passing or failing program execution. We also present results from an initial feasibility study, based on multiple versions of a software subject, in which we investigate several issues vital to the applicability of the technique. Finally, we present some lessons learned regarding the interplay between the reliability of classification models and the amount and type of data collected.