Optimally profiling and tracing programs
POPL '92 Proceedings of the 19th ACM SIGPLAN-SIGACT symposium on Principles of programming languages
Optimally profiling and tracing programs
ACM Transactions on Programming Languages and Systems (TOPLAS)
Exploiting hardware performance counters with flow and context sensitive profiling
Proceedings of the ACM SIGPLAN 1997 conference on Programming language design and implementation
MICRO 30 Proceedings of the 30th annual ACM/IEEE international symposium on Microarchitecture
ESEC '97/FSE-5 Proceedings of the 6th European SOFTWARE ENGINEERING conference held jointly with the 5th ACM SIGSOFT international symposium on Foundations of software engineering
An approach to large-scale collection of application usage data over the Internet
Proceedings of the 20th international conference on Software engineering
Residual test coverage monitoring
Proceedings of the 21st international conference on Software engineering
Multivariate visualization in observation-based testing
Proceedings of the 22nd international conference on Software engineering
Dynamically Discovering Likely Program Invariants to Support Program Evolution
IEEE Transactions on Software Engineering - Special issue on 1999 international conference on software engineering
A framework for reducing the cost of instrumented code
Proceedings of the ACM SIGPLAN 2001 conference on Programming language design and implementation
ICSE '01 Proceedings of the 23rd International Conference on Software Engineering
Finding failures by cluster analysis of execution profiles
ICSE '01 Proceedings of the 23rd International Conference on Software Engineering
Test Case Prioritization: A Family of Empirical Studies
IEEE Transactions on Software Engineering
Exploring Expect
Software Reliability Engineered Testing
Software Reliability Engineered Testing
Gamma system: continuous evolution of software after deployment
ISSTA '02 Proceedings of the 2002 ACM SIGSOFT international symposium on Software testing and analysis
Monitoring deployed software using software tomography
Proceedings of the 2002 ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering
Bug isolation via remote program sampling
PLDI '03 Proceedings of the ACM SIGPLAN 2003 conference on Programming language design and implementation
Gprof: A call graph execution profiler
SIGPLAN '82 Proceedings of the 1982 SIGPLAN symposium on Compiler construction
Leveraging field data for impact analysis and regression testing
Proceedings of the 9th European software engineering conference held jointly with 11th ACM SIGSOFT international symposium on Foundations of software engineering
Anomalies as Precursors of Field Failures
ISSRE '03 Proceedings of the 14th International Symposium on Software Reliability Engineering
Skoll: Distributed Continuous Quality Assurance
Proceedings of the 26th International Conference on Software Engineering
Covering arrays for efficient fault characterization in complex configuration spaces
ISSTA '04 Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis
ISSTA '04 Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis
Active learning for automatic classification of software behavior
ISSTA '04 Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis
A historical perspective on runtime assertion checking in software development
ACM SIGSOFT Software Engineering Notes
Experimental program analysis: a new paradigm for program analysis
Proceedings of the 28th international conference on Software engineering
Experimental program analysis: a new program analysis paradigm
Proceedings of the 2006 international symposium on Software testing and analysis
Adaptive Online Program Analysis
ICSE '07 Proceedings of the 29th international conference on Software Engineering
A Technique for Enabling and Supporting Debugging of Field Failures
ICSE '07 Proceedings of the 29th international conference on Software Engineering
Software Testing Research: Achievements, Challenges, Dreams
FOSE '07 2007 Future of Software Engineering
Techniques for Classifying Executions of Deployed Software to Support Software Engineering Tasks
IEEE Transactions on Software Engineering
Analysis of a deployed software
Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
Analysis of a deployed software
The 6th Joint Meeting on European software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering: companion papers
An approach to detecting duplicate bug reports using natural language and execution information
Proceedings of the 30th international conference on Software engineering
Using machine learning to refine Category-Partition test specifications and test suites
Information and Software Technology
Reducing the Cost of Path Property Monitoring Through Sampling
ASE '08 Proceedings of the 2008 23rd IEEE/ACM International Conference on Automated Software Engineering
Information and Software Technology
Architectural verification of black-box component-based systems
RISE'06 Proceedings of the 3rd international conference on Rapid integration of software engineering techniques
Property-aware program sampling
Proceedings of the 9th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering
Monitoring, analysis, and testing of deployed software
Proceedings of the FSE/SDP workshop on Future of software engineering research
Camouflage: automated anonymization of field data
Proceedings of the 33rd International Conference on Software Engineering
Characterizing the differences between pre- and post- release versions of software
Proceedings of the 33rd International Conference on Software Engineering
Access policy compliance testing in a user centric trust service infrastructure
Proceedings of the International Workshop on Quality Assurance for Service-Based Applications
Lattice-Based Sampling for Path Property Monitoring
ACM Transactions on Software Engineering and Methodology (TOSEM)
BugRedux: reproducing field failures for in-house debugging
Proceedings of the 34th International Conference on Software Engineering
How much does unused code matter for maintenance?
Proceedings of the 34th International Conference on Software Engineering
F3: fault localization for field failures
Proceedings of the 2013 International Symposium on Software Testing and Analysis
Performance optimization of deployed software-as-a-service applications
Journal of Systems and Software
Hi-index | 0.00 |
An understanding of how software is employed in the field can yield many opportunities for quality improvements. Profiling released software can provide such an understanding. However, profiling released software is difficult due to the potentially large number of deployed sites that must be profiled, the transparency requirements at a user's site, and the remote data collection and deployment management process. Researchers have recently proposed various approaches to tap into the opportunities offered by profiling deployed systems and overcome those challenges. Initial studies have illustrated the application of these approaches and have shown their feasibility. Still, the proposed approaches, and the tradeoffs between overhead, accuracy, and potential benefits for the testing activity have been barely quantified. This paper aims to overcome those limitations. Our analysis of 1,200 user sessions on a 155 KLOC deployed system substantiates the ability of field data to support test suite improvements, assesses the efficiency of profiling techniques for released software, and the effectiveness of testing efforts that leverage profiled field data.