Program testing by specification mutation
Computer Languages
Selecting Software Test Data Using Data Flow Information
IEEE Transactions on Software Engineering
Experimentation in software engineering
IEEE Transactions on Software Engineering
The evaluation of program-based software test data adequacy criteria
Communications of the ACM
The category-partition method for specifying and generating fuctional tests
Communications of the ACM
TAV3 Proceedings of the ACM SIGSOFT '89 third symposium on Software testing, analysis, and verification
Software testing techniques (2nd ed.)
Software testing techniques (2nd ed.)
Selecting tests and identifying test coverage requirements for modified software
ISSTA '94 Proceedings of the 1994 ACM SIGSOFT international symposium on Software testing and analysis
Experimental evaluation in computer science: a quantitative study
Journal of Systems and Software
An experimental determination of sufficient mutant operators
ACM Transactions on Software Engineering and Methodology (TOSEM)
Analyzing Regression Test Selection Techniques
IEEE Transactions on Software Engineering
A safe, efficient regression test selection technique
ACM Transactions on Software Engineering and Methodology (TOSEM)
Experiments of the effectiveness of dataflow- and controlflow-based test adequacy criteria
ICSE '94 Proceedings of the 16th international conference on Software engineering
TestTube: a system for selective regression testing
ICSE '94 Proceedings of the 16th international conference on Software engineering
Semantics Guided Regression Test Cost Reduction
IEEE Transactions on Software Engineering
Regression testing in an industrial environment
Communications of the ACM
Software test automation: effective use of test execution tools
Software test automation: effective use of test execution tools
Building Knowledge through Families of Experiments
IEEE Transactions on Software Engineering
Experimentation in software engineering: an introduction
Experimentation in software engineering: an introduction
Quickly detecting relevant program invariants
Proceedings of the 22nd international conference on Software engineering
A comparative study of coarse- and fine-grained safe regression test-selection techniques
ACM Transactions on Software Engineering and Methodology (TOSEM)
Interface Mutation: An Approach for Integration Testing
IEEE Transactions on Software Engineering
Incorporating varying test costs and fault severities into test case prioritization
ICSE '01 Proceedings of the 23rd International Conference on Software Engineering
Using symbolic execution for verifying safety-critical systems
Proceedings of the 8th European software engineering conference held jointly with 9th ACM SIGSOFT international symposium on Foundations of software engineering
Prioritizing Test Cases For Regression Testing
IEEE Transactions on Software Engineering
Proceedings of the 24th International Conference on Software Engineering
The impact of test suite granularity on the cost-effectiveness of regression testing
Proceedings of the 24th International Conference on Software Engineering
Techniques for Selective Revalidation
IEEE Software
Case Studies for Method and Tool Evaluation
IEEE Software
An Experimental Comparison of the Effectiveness of Branch Testing and Data Flow Testing
IEEE Transactions on Software Engineering
Preliminary guidelines for empirical research in software engineering
IEEE Transactions on Software Engineering
Test Set Size Minimization and Fault Detection Effectiveness: A Case Study in a Space Application
COMPSAC '97 Proceedings of the 21st International Computer Software and Applications Conference
Improving test suites via operational abstraction
Proceedings of the 25th International Conference on Software Engineering
The Impact of Software Evolution on Code Coverage Information
ICSM '01 Proceedings of the IEEE International Conference on Software Maintenance (ICSM'01)
An Implementation of and Experiment with Semantic Differencing
ICSM '01 Proceedings of the IEEE International Conference on Software Maintenance (ICSM'01)
Empirical Evaluation of the Textual Differencing Regression Testing Technique
ICSM '98 Proceedings of the International Conference on Software Maintenance
A Study of Effective Regression Testing in Practice
ISSRE '97 Proceedings of the Eighth International Symposium on Software Reliability Engineering
Modeling the Cost-Benefits Tradeoffs for Regression Testing Techniques
ICSM '02 Proceedings of the International Conference on Software Maintenance (ICSM'02)
Using Spanning Sets for Coverage Testing
IEEE Transactions on Software Engineering
Reviewing 25 Years of Testing Technique Experiments
Empirical Software Engineering
Empirical Studies of Test Case Prioritization in a JUnit Testing Environment
ISSRE '04 Proceedings of the 15th International Symposium on Software Reliability Engineering
Is mutation an appropriate tool for testing experiments?
Proceedings of the 27th international conference on Software engineering
A Controlled Experiment Assessing Test Case Prioritization Techniques via Mutation Faults
ICSM '05 Proceedings of the 21st IEEE International Conference on Software Maintenance
Prioritizing JUnit Test Cases: An Empirical Assessment and Cost-Benefits Analysis
Empirical Software Engineering
Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
Failure proximity: a fault localization-based approach
Proceedings of the 14th ACM SIGSOFT international symposium on Foundations of software engineering
Proceedings of the 14th ACM SIGSOFT international symposium on Foundations of software engineering
Carving differential unit test cases from system test cases
Proceedings of the 14th ACM SIGSOFT international symposium on Foundations of software engineering
Automated Fault Localization for C Programs
Electronic Notes in Theoretical Computer Science (ENTCS)
On the Use of Mutation Faults in Empirical Assessments of Test Case Prioritization Techniques
IEEE Transactions on Software Engineering
Proceedings of the 2007 ACM SIGPLAN conference on Programming language design and implementation
Software Testing Research: Achievements, Challenges, Dreams
FOSE '07 2007 Future of Software Engineering
Search Algorithms for Regression Test Case Prioritization
IEEE Transactions on Software Engineering
Statistical debugging using compound boolean predicates
Proceedings of the 2007 international symposium on Software testing and analysis
Proceedings of the 2007 international symposium on Software testing and analysis
Pareto efficient multi-objective test case selection
Proceedings of the 2007 international symposium on Software testing and analysis
Proceedings of the 2007 international symposium on Software testing and analysis
Differential testing: a new approach to change detection
Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
Trace anomalies as precursors of field failures: an empirical study
Empirical Software Engineering
Empirical studies in reverse engineering: state of the art and future trends
Empirical Software Engineering
Differential testing: a new approach to change detection
The 6th Joint Meeting on European software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering: companion papers
Extraction of bug localization benchmarks from history
Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering
Database-aware test coverage monitoring
ISEC '08 Proceedings of the 1st India software engineering conference
DSD-Crasher: A hybrid analysis tool for bug finding
ACM Transactions on Software Engineering and Methodology (TOSEM)
PHALANX: a graph-theoretic framework for test case prioritization
Proceedings of the 2008 ACM symposium on Applied computing
Time will tell: fault localization using time spectra
Proceedings of the 30th international conference on Software engineering
Incremental state-space exploration for programs with dynamically allocated data
Proceedings of the 30th international conference on Software engineering
Formal concept analysis applied to fault localization
Companion of the 30th international conference on Software engineering
Using sensitivity analysis to create simplified economic models for regression testing
ISSTA '08 Proceedings of the 2008 international symposium on Software testing and analysis
Configuration-aware regression testing: an empirical study of sampling and prioritization
ISSTA '08 Proceedings of the 2008 international symposium on Software testing and analysis
AFID: an automated fault identification tool
ISSTA '08 Proceedings of the 2008 international symposium on Software testing and analysis
The probabilistic program dependence graph and its application to fault diagnosis
ISSTA '08 Proceedings of the 2008 international symposium on Software testing and analysis
Statistical Debugging Using Latent Topic Models
ECML '07 Proceedings of the 18th European conference on Machine Learning
Profile-guided program simplification for effective testing and analysis
Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering
An empirical study of the effect of time constraints on the cost-benefits of regression testing
Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering
Software Fault Localization Using N-gram Analysis
WASA '08 Proceedings of the Third International Conference on Wireless Algorithms, Systems, and Applications
Effective identification of failure-inducing changes: a hybrid approach
Proceedings of the 8th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering
Testing-based Process for Evaluating Component Replaceability
Electronic Notes in Theoretical Computer Science (ENTCS)
Empirical evaluation of a nesting testability transformation for evolutionary testing
ACM Transactions on Software Engineering and Methodology (TOSEM)
Algorithms for Automatically Computing the Causal Paths of Failures
FASE '09 Proceedings of the 12th International Conference on Fundamental Approaches to Software Engineering: Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2009
ICSE '09 Proceedings of the 31st International Conference on Software Engineering
Safe-commit analysis to facilitate team software development
ICSE '09 Proceedings of the 31st International Conference on Software Engineering
An empirical study of the factors that reduce the effectiveness of coverage-based fault localization
Proceedings of the 2nd International Workshop on Defects in Large Software Systems: Held in conjunction with the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2009)
Information and Software Technology
Measuring the strength of information flows in programs
ACM Transactions on Software Engineering and Methodology (TOSEM)
Proceedings of the eighteenth international symposium on Software testing and analysis
Darwin: an approach for debugging evolving programs
Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
Capturing propagation of infected program states
Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
Is non-parametric hypothesis testing model robust for statistical fault localization?
Information and Software Technology
A practical evaluation of spectrum-based fault localization
Journal of Systems and Software
Increasing Test Granularity by Aggregating Unit Tests
ASE '08 Proceedings of the 2008 23rd IEEE/ACM International Conference on Automated Software Engineering
DiffGen: Automated Regression Unit-Test Generation
ASE '08 Proceedings of the 2008 23rd IEEE/ACM International Conference on Automated Software Engineering
A systematic review on regression test selection techniques
Information and Software Technology
Test input reduction for result inspection to facilitate fault localization
Automated Software Engineering
Fault localization through evaluation sequences
Journal of Systems and Software
Using hybrid algorithm for Pareto efficient multi-objective test suite minimisation
Journal of Systems and Software
A prioritization approach for software test cases based on Bayesian networks
FASE'07 Proceedings of the 10th international conference on Fundamental approaches to software engineering
Contextual integration testing of classes
FASE'08/ETAPS'08 Proceedings of the Theory and practice of software, 11th international conference on Fundamental approaches to software engineering
Property-aware program sampling
Proceedings of the 9th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering
Towards a unified fault-detection benchmark
Proceedings of the 9th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering
Structural coverage of feasible code
Proceedings of the 5th Workshop on Automation of Software Test
AFID: an automated approach to collecting software faults
Automated Software Engineering
Factors affecting the use of genetic algorithms in test suite augmentation
Proceedings of the 12th annual conference on Genetic and evolutionary computation
Causal inference for statistical fault localization
Proceedings of the 19th international symposium on Software testing and analysis
Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement
Test generation to expose changes in evolving programs
Proceedings of the IEEE/ACM international conference on Automated software engineering
Exploiting count spectra for Bayesian fault localization
Proceedings of the 6th International Conference on Predictive Models in Software Engineering
Diagnosing multiple intermittent failures using maximum likelihood estimation
Artificial Intelligence
Combining hardware and software instrumentation to classify program executions
Proceedings of the eighteenth ACM SIGSOFT international symposium on Foundations of software engineering
Directed test suite augmentation: techniques and tradeoffs
Proceedings of the eighteenth ACM SIGSOFT international symposium on Foundations of software engineering
Proceedings of the FSE/SDP workshop on Future of software engineering research
Validity concerns in software engineering research
Proceedings of the FSE/SDP workshop on Future of software engineering research
Proceedings of the 2010 Conference of the Center for Advanced Studies on Collaborative Research
Execution-aware fault localization based on the control flow analysis
ICICA'10 Proceedings of the First international conference on Information computing and applications
Non-parametric statistical fault localization
Journal of Systems and Software
A test suite reduction approach based on pairwise interaction of requirements
Proceedings of the 2011 ACM Symposium on Applied Computing
An empirical study on the effectiveness of time-aware test case prioritization techniques
Proceedings of the 2011 ACM Symposium on Applied Computing
Scalable graph analyzing approach for software fault-localization
Proceedings of the 6th International Workshop on Automation of Software Test
Diagnosing new faults using mutants and prior faults (NIER track)
Proceedings of the 33rd International Conference on Software Engineering
Probabilistic points-to analysis for java
CC'11/ETAPS'11 Proceedings of the 20th international conference on Compiler construction: part of the joint European conferences on theory and practice of software
Cause clue clauses: error localization using maximum satisfiability
Proceedings of the 32nd ACM SIGPLAN conference on Programming language design and implementation
A model for spectra-based software diagnosis
ACM Transactions on Software Engineering and Methodology (TOSEM)
eXpress: guided path exploration for efficient regression test generation
Proceedings of the 2011 International Symposium on Software Testing and Analysis
Change-aware preemption prioritization
Proceedings of the 2011 International Symposium on Software Testing and Analysis
Are automated debugging techniques actually helping programmers?
Proceedings of the 2011 International Symposium on Software Testing and Analysis
Minimizing reproduction of software failures
Proceedings of the 2011 International Symposium on Software Testing and Analysis
An evaluation of change-based coverage criteria
Proceedings of the 10th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools
Mitigating the confounding effects of program dependences for effective fault localization
Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering
Strong higher order mutation-based test data generation
Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering
Path exploration based on symbolic output
Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering
SCORE: a scalable concolic testing tool for reliable embedded software
Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering
Bug-Assist: assisting fault localization in ANSI-C programs
CAV'11 Proceedings of the 23rd international conference on Computer aided verification
Enhancing structural software coverage by incrementally computing branch executability
Software Quality Control
A variable strength interaction test suites generation strategy using Particle Swarm Optimization
Journal of Systems and Software
Elementary landscape decomposition of the test suite minimization problem
SSBSE'11 Proceedings of the Third international conference on Search based software engineering
Highly scalable multi objective test suite minimisation using graphics cards
SSBSE'11 Proceedings of the Third international conference on Search based software engineering
Journal of Computer and System Sciences
Prioritizing test cases with string distances
Automated Software Engineering
Leveraging Strength-Based Dynamic Information Flow Analysis to Enhance Data Value Prediction
ACM Transactions on Architecture and Code Optimization (TACO)
Test case filtering and prioritization based on coverage of combinations of program elements
WODA '09 Proceedings of the Seventh International Workshop on Dynamic Analysis
Automated error localization and correction for imperative programs
Proceedings of the International Conference on Formal Methods in Computer-Aided Design
Combining search-based and constraint-based testing
ASE '11 Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering
Fault-localization using dynamic slicing and change impact analysis
ASE '11 Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering
Prioritizing tests for fault localization through ambiguity group reduction
ASE '11 Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering
Using entropy measures for comparison of software traces
Information Sciences: an International Journal
Reduction of test suites using mutation
FASE'12 Proceedings of the 15th international conference on Fundamental Approaches to Software Engineering
Regression testing minimization, selection and prioritization: a survey
Software Testing, Verification & Reliability
Multi-objective coevolutionary automated software correction
Proceedings of the 14th annual conference on Genetic and evolutionary computation
Evaluating program analysis and testing tools with the RUGRAT random benchmark application generator
Proceedings of the 2012 Workshop on Dynamic Analysis
On-demand test suite reduction
Proceedings of the 34th International Conference on Software Engineering
Multi-label software behavior learning
Proceedings of the 34th International Conference on Software Engineering
Practical isolation of failure-inducing changes for debugging regression faults
Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering
Diversity maximization speedup for fault localization
Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering
An improved method for test case prioritization by incorporating historical test case data
Science of Computer Programming
AI for the win: improving spectrum-based fault localization
ACM SIGSOFT Software Engineering Notes
Semantic fault diagnosis: automatic natural-language fault descriptions
Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering
Concept-based failure clustering
Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering
Testing-based process for component substitutability
Software Testing, Verification & Reliability
On the application of SAT solvers to the test suite minimization problem
SSBSE'12 Proceedings of the 4th international conference on Search Based Software Engineering
Evolving human competitive spectra-based fault localisation techniques
SSBSE'12 Proceedings of the 4th international conference on Search Based Software Engineering
Science of Computer Programming
A general noise-reduction framework for fault localization of Java programs
Information and Software Technology
Information and Software Technology
Exploring program phases for statistical bug localization
Proceedings of the 11th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering
The optimisation of stochastic grammars to enable cost-effective probabilistic structural testing
Proceedings of the 15th annual conference on Genetic and evolutionary computation
Testing techniques selection based on ODC fault types and software metrics
Journal of Systems and Software
Mutation testing strategies using mutant classification
Proceedings of the 28th Annual ACM Symposium on Applied Computing
Spectral debugging: how much better can we do?
ACSC '12 Proceedings of the Thirty-fifth Australasian Computer Science Conference - Volume 122
Comparing non-adequate test suites using coverage criteria
Proceedings of the 2013 International Symposium on Software Testing and Analysis
Selective mutation testing for concurrent code
Proceedings of the 2013 International Symposium on Software Testing and Analysis
Orthogonal exploration of the search space in evolutionary test case generation
Proceedings of the 2013 International Symposium on Software Testing and Analysis
Improving fault tolerance utilizing hardware-software-co-synthesis
Proceedings of the Conference on Design, Automation and Test in Europe
Bridging the gap between the total and additional test-case prioritization strategies
Proceedings of the 2013 International Conference on Software Engineering
Partition-based regression verification
Proceedings of the 2013 International Conference on Software Engineering
SemFix: program repair via semantic analysis
Proceedings of the 2013 International Conference on Software Engineering
Quantitative program slicing: separating statements by relevance
Proceedings of the 2013 International Conference on Software Engineering
DUA-forensics: a fine-grained dependence analysis and instrumentation framework based on Soot
Proceedings of the 2nd ACM SIGPLAN International Workshop on State Of the Art in Java Program analysis
Enhancing symbolic execution with built-in term rewriting and constrained lazy initialization
Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering
Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering
Differential assertion checking
Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering
Fault localization prioritization: Comparing information-theoretic and coverage-based approaches
ACM Transactions on Software Engineering and Methodology (TOSEM) - In memoriam, fault detection and localization, formal methods, modeling and design
Test case prioritization: a systematic mapping study
Software Quality Control
Injecting mechanical faults to localize developer faults for evolving software
Proceedings of the 2013 ACM SIGPLAN international conference on Object oriented programming systems languages & applications
Path exploration based on symbolic output
ACM Transactions on Software Engineering and Methodology (TOSEM) - Testing, debugging, and error handling, formal methods, lifecycle concerns, evolution and maintenance
BPELDebugger: An effective BPEL-specific fault localization framework
Information and Software Technology
Prevalence of coincidental correctness and mitigation of its impact on fault localization
ACM Transactions on Software Engineering and Methodology (TOSEM)
Safely reducing the cost of unit level symbolic execution through read/write analysis
ACM SIGSOFT Software Engineering Notes
Generating profile-based signatures for online intrusion and failure detection
Information and Software Technology
An approach to testing commercial embedded systems
Journal of Systems and Software
Static test case prioritization using topic models
Empirical Software Engineering
Generating test data for both path coverage and fault detection using genetic algorithms
Frontiers of Computer Science: Selected Publications from Chinese Universities
HSFal: Effective fault localization using hybrid spectrum of full slices and execution slices
Journal of Systems and Software
An empirical study on the use of mutant traces for diagnosis of faults in deployed systems
Journal of Systems and Software
Hi-index | 0.00 |
Where the creation, understanding, and assessment of software testing and regression testing techniques are concerned, controlled experimentation is an indispensable research methodology. Obtaining the infrastructure necessary to support such experimentation, however, is difficult and expensive. As a result, progress in experimentation with testing techniques has been slow, and empirical data on the costs and effectiveness of techniques remains relatively scarce. To help address this problem, we have been designing and constructing infrastructure to support controlled experimentation with testing and regression testing techniques. This paper reports on the challenges faced by researchers experimenting with testing techniques, including those that inform the design of our infrastructure. The paper then describes the infrastructure that we are creating in response to these challenges, and that we are now making available to other researchers, and discusses the impact that this infrastructure has had and can be expected to have.