Experimentation in software engineering
IEEE Transactions on Software Engineering
Comparing the Effectiveness of Software Testing Strategies
IEEE Transactions on Software Engineering
Software engineering standards
Software engineering standards
A scientific methodology for MIS case studies
MIS Quarterly
Science and Substance: A Challenge to Software Engineers
IEEE Software
Experimental evaluation in computer science: a quantitative study
Journal of Systems and Software
Software metrics (2nd ed.): a rigorous and practical approach
Software metrics (2nd ed.): a rigorous and practical approach
Evolving and packaging reading technologies
Journal of Systems and Software - Special issue on achieving quality in software
Reinventing methodology: who reads it and why?
Communications of the ACM
The Psychological Study of Programming
ACM Computing Surveys (CSUR)
Studying programmer behavior experimentally: the problems of proper methodology
Communications of the ACM
A Replicated Experiment to Assess Requirements InspectionTechniques
Empirical Software Engineering
Empirical Evaluation of Software Maintenance Technologies
Empirical Software Engineering
Further Experiences with Scenarios and Checklists
Empirical Software Engineering
Empirical Software Engineering
An Extended Replication of an Experiment for AssessingMethods for Software Requirements Inspections
Empirical Software Engineering
Comparing Detection Methods for Software Requirements Inspections: A Replicated Experiment
IEEE Transactions on Software Engineering
Communication and Organization: An Empirical Study of Discussion in Inspection Meetings
IEEE Transactions on Software Engineering
The Experimental Paradigm in Software Engineering
Proceedings of the International Workshop on Experimental Software Engineering Issues: Critical Assessment and Future Directions
Experimenting with Error Abstraction in Requirements Documents
METRICS '98 Proceedings of the 5th International Symposium on Software Metrics
Investigating Maintenance Processes in a Framework-Based Environment
ICSM '98 Proceedings of the International Conference on Software Maintenance
Developing techniques for using software documents: a series of empirical studies
Developing techniques for using software documents: a series of empirical studies
Design and Analysis of Experiments
Design and Analysis of Experiments
An empirical methodology for introducing software processes
Proceedings of the 8th European software engineering conference held jointly with 9th ACM SIGSOFT international symposium on Foundations of software engineering
Empirical Evaluation of CASE Tools Usage at Nokia
Empirical Software Engineering
Designing and Conducting an Empirical Study on Test Management Automation
Empirical Software Engineering
A Controlled Experiment for Validating Class Diagram Structural Complexity Metrics
OOIS '02 Proceedings of the 8th International Conference on Object-Oriented. Information Systems
Using Metrics to Predict OO Information Systems Maintainability
CAiSE '01 Proceedings of the 13th International Conference on Advanced Information Systems Engineering
New directions on agile methods: a comparative analysis
Proceedings of the 25th International Conference on Software Engineering
Replicated studies: building a body of knowledge about software reading techniques
Lecture notes on empirical software engineering
An Experimental Comparison of Usage-Based and Checklist-Based Reading
IEEE Transactions on Software Engineering
Metrics for data warehouse quality
Effective databases for text & document management
Evaluation of Usage-Based Reading—Conclusions after Three Experiments
Empirical Software Engineering
Knowledge-Sharing Issues in Experimental Software Engineering
Empirical Software Engineering
Empirical Analysis of Safety-Critical Anomalies During Operations
IEEE Transactions on Software Engineering
Meta-knowledge in systems design: panacea … or undelivered promise?
The Knowledge Engineering Review
Toward a Software Testing and Reliability Early Warning Metric Suite
Proceedings of the 26th International Conference on Software Engineering
Statistical significance testing: a panacea for software technology experiments?
Journal of Systems and Software - Special issue: Applications of statistics in software engineering
IEEE Transactions on Software Engineering
Collecting Feedback during Software Engineering Experiments
Empirical Software Engineering
Use of relative code churn measures to predict system defect density
Proceedings of the 27th international conference on Software engineering
Experimental context classification: incentives and experience of subjects
Proceedings of the 27th international conference on Software engineering
Empirical validation of pair programming
Proceedings of the 27th international conference on Software engineering
Confirming the influence of educational background in pair-design knowledge through experiments
Proceedings of the 2005 ACM symposium on Applied computing
Staffing a software project: a constraint satisfaction approach
EDSER '05 Proceedings of the seventh international workshop on Economics-driven software engineering research
REBSE '05 Proceedings of the 2005 workshop on Realising evidence-based software engineering
Empirical Software Engineering
Improvement of Design Specifications with Inspection and Testing
EUROMICRO '05 Proceedings of the 31st EUROMICRO Conference on Software Engineering and Advanced Applications
A Survey of Controlled Experiments in Software Engineering
IEEE Transactions on Software Engineering
Two controlled experiments concerning the comparison of pair programming to peer review
Journal of Systems and Software
On the success of empirical studies in the international conference on software engineering
Proceedings of the 28th international conference on Software engineering
Mining metrics to predict component failures
Proceedings of the 28th international conference on Software engineering
FMESP: framework for the modeling and evaluation of software processes
Proceedings of the 2004 workshop on Quantitative techniques for software agile process
Motivations and measurements in an agile case study
Proceedings of the 2004 workshop on Quantitative techniques for software agile process
Analysis of the influence of communication between researchers on experiment replication
Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
Using observational pilot studies to test and improve lab packages
Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
Evaluating the efficacy of test-driven development: industrial case studies
Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
Journal of Systems and Software - Special issue: Quality software
FMESP: framework for the modeling and evaluation of software processes
Journal of Systems Architecture: the EUROMICRO Journal - Special issue: AGILE methodologies for software production
Motivations and measurements in an agile case study
Journal of Systems Architecture: the EUROMICRO Journal - Special issue: AGILE methodologies for software production
Evaluating performances of pair designing in industry
Journal of Systems and Software
Scientific research ontology to support systematic review in software engineering
Advanced Engineering Informatics
The Impact of UML Documentation on Software Maintenance: An Experimental Evaluation
IEEE Transactions on Software Engineering
The Future of Empirical Methods in Software Engineering Research
FOSE '07 2007 Future of Software Engineering
Metrics for data warehouse conceptual models understandability
Information and Software Technology
Difficulties experienced by students in maintaining object-oriented systems: an empirical study
ACE '07 Proceedings of the ninth Australasian conference on Computing education - Volume 66
An analysis of developers' tasks using low-level, automatically collected data
Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
Empirical studies in reverse engineering: state of the art and future trends
Empirical Software Engineering
Building measure-based prediction models for UML class diagram maintainability
Empirical Software Engineering
Evaluating quality of conceptual modelling scripts based on user perceptions
Data & Knowledge Engineering
An analysis of developers' tasks using low-level, automatically collected data
The 6th Joint Meeting on European software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering: companion papers
Creativity support tools: accelerating discovery and innovation
Communications of the ACM
Defining and validating metrics for assessing the understandability of entity-relationship diagrams
Data & Knowledge Engineering
A preliminary study on various implementation approaches of domain-specific language
Information and Software Technology
Staffing a software project: A constraint satisfaction and optimization-based approach
Computers and Operations Research
The role of replications in Empirical Software Engineering
Empirical Software Engineering
On the difficulty of replicating human subjects studies in software engineering
Proceedings of the 30th international conference on Software engineering
The influence of organizational structure on software quality: an empirical case study
Proceedings of the 30th international conference on Software engineering
Predicting defects using network analysis on dependency graphs
Proceedings of the 30th international conference on Software engineering
Empirical Software Engineering
Software engineering article types: An analysis of the literature
Journal of Systems and Software
Pair programming: what's in it for me?
Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement
Experiences on Using Software Experiments in the Validation of Industrial Research Questions
Software Process and Product Measurement
Empirical Analysis of a Distributed Software Development Project
Balancing Agility and Formalism in Software Engineering
MoDELS '08 Proceedings of the 11th international conference on Model Driven Engineering Languages and Systems
Can developer-module networks predict failures?
Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering
Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering
The effect of task order on the maintainability of object-oriented software
Information and Software Technology
Knowledge Base and Experience Factory for Empowering Competitiveness
Software Engineering
An experimental investigation of personality types impact on pair effectiveness in pair programming
Empirical Software Engineering
Journal of Systems and Software
Empirical Validation of Measures for UML Class Diagrams: A Meta-Analysis Study
Models in Software Engineering
An exploratory study on China's software engineers' unpaid overtime working
Proceedings of the special interest group on management information system's 47th annual conference on Computer personnel research
Synthesis, Analysis, and Modeling of Large-Scale Mission-Critical Embedded Software Systems
ICSP '09 Proceedings of the International Conference on Software Process: Trustworthy Software Development Processes
Cross-project defect prediction: a large scale experiment on data vs. domain vs. process
Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
Empirical Software Engineering
Guidelines on the aesthetic quality of UML class diagrams
Information and Software Technology
Proceedings of the 24th ACM SIGPLAN conference companion on Object oriented programming systems languages and applications
Exploring traceability-based requirements inspection
SEA '07 Proceedings of the 11th IASTED International Conference on Software Engineering and Applications
An empirical study of the effects of personality in pair programming using the five-factor model
ESEM '09 Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement
Test coverage and post-verification defects: A multiple case study
ESEM '09 Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement
Predicting defects with program dependencies
ESEM '09 Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement
Simulating families of studies to build confidence in defect hypotheses
Information and Software Technology
Assessing multiview framework (MF) comprehensibility and efficiency: A replicated experiment
Information and Software Technology
Information needs in bug reports: improving cooperation between developers and users
Proceedings of the 2010 ACM conference on Computer supported cooperative work
The impact of structural complexity on the understandability of UML statechart diagrams
Information Sciences: an International Journal
No-redundant metrics for UML class diagram structural complexity
CAiSE'03 Proceedings of the 15th international conference on Advanced information systems engineering
Measuring the quality of entity relationship diagrams
ER'00 Proceedings of the 19th international conference on Conceptual modeling
The role of controlled experiments in software engineering research
Proceedings of the 2006 international conference on Empirical software engineering issues: critical assessment and future directions
Potential of open source systems as project repositories for empirical studies working group results
Proceedings of the 2006 international conference on Empirical software engineering issues: critical assessment and future directions
Empirical software engineering: teaching methods and conducting studies
Proceedings of the 2006 international conference on Empirical software engineering issues: critical assessment and future directions
Roadmapping working group 4 results
Proceedings of the 2006 international conference on Empirical software engineering issues: critical assessment and future directions
ER'07 Proceedings of the 2007 conference on Advances in conceptual modeling: foundations and applications
On the effectiveness of unit test automation at Microsoft
ISSRE'09 Proceedings of the 20th IEEE international conference on software reliability engineering
Characterizing and predicting which bugs get fixed: an empirical study of Microsoft Windows
Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 1
Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 1
Penalty policies in professional software development practice: a multi-method field study
Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2
Information Sciences: an International Journal
Replications types in experimental disciplines
Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement
Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement
The effects of neuroticism on pair programming: an empirical study in the higher education context
Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement
Investigating the use of tags in collaborative development environments: a replicated study
Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement
The beauty and the beast: vulnerabilities in red hat’s packages
USENIX'09 Proceedings of the 2009 conference on USENIX Annual technical conference
Information and Software Technology
Analytics for software development
Proceedings of the FSE/SDP workshop on Future of software engineering research
Myths in software engineering: from the other side
TAP'10 Proceedings of the 4th international conference on Tests and proofs
Testing techniques in software engineering
Testing techniques in software engineering
Graphical versus textual software measurement modelling: an empirical study
Software Quality Control
"Not my bug!" and other reasons for software bug report reassignments
Proceedings of the ACM 2011 conference on Computer supported cooperative work
Impact of test-driven development on productivity, code and tests: A controlled experiment
Information and Software Technology
Empirical Software Engineering
Comparing fine-grained source code changes and code churn for bug prediction
Proceedings of the 8th Working Conference on Mining Software Repositories
Proceedings of the 6th International Workshop on Traceability in Emerging Forms of Software Engineering
Evaluating requirements modeling methods based on user perceptions: A family of experiments
Information Sciences: an International Journal
Assessing the maintainability of software product line feature models using structural metrics
Software Quality Control
Using web objects for development effort estimation of web applications: a replicated study
PROFES'11 Proceedings of the 12th international conference on Product-focused software process improvement
PROFES'11 Proceedings of the 12th international conference on Product-focused software process improvement
Information and Software Technology
A precise style for business process modelling: results from two controlled experiments
Proceedings of the 14th international conference on Model driven engineering languages and systems
ADVIS'04 Proceedings of the Third international conference on Advances in Information Systems
Lessons learned from an XP experiment with students: test-first needs more teachings
PROFES'06 Proceedings of the 7th international conference on Product-Focused Software Process Improvement
WM'05 Proceedings of the Third Biennial conference on Professional Knowledge Management
Pair programming vs. side-by-side programming
EuroSPI'05 Proceedings of the 12th European conference on Software Process Improvement
Evolving defect "folklore": a cross-study analysis of software defect behavior
SPW'05 Proceedings of the 2005 international conference on Unifying the Software Process Spectrum
Empirical Software Engineering
Empirical study on the productivity of the pair programming
XP'05 Proceedings of the 6th international conference on Extreme Programming and Agile Processes in Software Engineering
Investigating the impact of active guidance on design inspection
PROFES'05 Proceedings of the 6th international conference on Product Focused Software Process Improvement
Evaluating the effect of composite states on the understandability of UML statechart diagrams
MoDELS'05 Proceedings of the 8th international conference on Model Driven Engineering Languages and Systems
Empirical Software Engineering
Empirical Software Engineering
Evaluating quality of conceptual models based on user perceptions
ER'06 Proceedings of the 25th international conference on Conceptual Modeling
An empirical study of the nesting level of composite states within UML statechart diagrams
ER'05 Proceedings of the 24th international conference on Perspectives in Conceptual Modeling
Factors of process model comprehension-Findings from a series of experiments
Decision Support Systems
FASE'06 Proceedings of the 9th international conference on Fundamental Approaches to Software Engineering
Replication of software engineering experiments
Empirical Software Engineering and Verification
Decision support for the software product line domain engineering lifecycle
Automated Software Engineering
Accounting for defect characteristics in evaluations of testing techniques
ACM Transactions on Software Engineering and Methodology (TOSEM)
Comparing inspection methods using controlled experiments
EASE'08 Proceedings of the 12th international conference on Evaluation and Assessment in Software Engineering
Empirical-WebGen: a web-based environment for the automatic generation of surveys and experiments
EASE'08 Proceedings of the 12th international conference on Evaluation and Assessment in Software Engineering
EASE'07 Proceedings of the 11th international conference on Evaluation and Assessment in Software Engineering
Does the level of detail of UML models affect the maintainability of source code?
MODELS'11 Proceedings of the 2011th international conference on Models in Software Engineering
Requirements engineering tools: Capabilities, survey and assessment
Information and Software Technology
Proceedings of the 34th International Conference on Software Engineering
Characterizing and predicting which bugs get reopened
Proceedings of the 34th International Conference on Software Engineering
Improving PSP education by pairing: an empirical study
Proceedings of the 34th International Conference on Software Engineering
Information and Software Technology
A note on rigour and replicability
ACM SIGSOFT Software Engineering Notes
What works for whom, where, when, and why?: on the role of context in empirical software engineering
Proceedings of the ACM-IEEE international symposium on Empirical software engineering and measurement
The effect of branching strategies on software quality
Proceedings of the ACM-IEEE international symposium on Empirical software engineering and measurement
Assessing the value of branches with what-if analysis
Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering
A field study of refactoring challenges and benefits
Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering
Empirical validation of a usability inspection method for model-driven Web development
Journal of Systems and Software
MODELS'12 Proceedings of the 15th international conference on Model Driven Engineering Languages and Systems
Comparing the usability of grid-bag and constraint-based layouts
Proceedings of the 24th Australian Computer-Human Interaction Conference
Enhancing design models with composition properties: a software maintenance study
Proceedings of the 12th annual international conference on Aspect-oriented software development
Proceedings of the 17th International Conference on Evaluation and Assessment in Software Engineering
Empirical evaluation of the effects of mixed project data on learning defect predictors
Information and Software Technology
Testing techniques selection based on ODC fault types and software metrics
Journal of Systems and Software
Expectations, outcomes, and challenges of modern code review
Proceedings of the 2013 International Conference on Software Engineering
Replicating mining studies with SOFAS
Proceedings of the 10th Working Conference on Mining Software Repositories
Evaluating applicability of combinatorial testing in an industrial environment: a case study
Proceedings of the 2013 International Workshop on Joining AcadeMiA and Industry Contributions to testing Automation
Diversity in software engineering research
Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering
The value of design rationale information
ACM Transactions on Software Engineering and Methodology (TOSEM) - In memoriam, fault detection and localization, formal methods, modeling and design
On the impact of UML analysis models on source-code comprehensibility and modifiability
ACM Transactions on Software Engineering and Methodology (TOSEM)
Information and Software Technology
An empirical approach for evaluating the usability of model-driven tools
Science of Computer Programming
Traceability and SysML design slices to support safety inspections: A controlled experiment
ACM Transactions on Software Engineering and Methodology (TOSEM)
Replications of software engineering experiments
Empirical Software Engineering
Hi-index | 0.02 |
Experimentation in software engineering is necessary but difficult. One reason is that there are a large number of context variables and, so, creating a cohesive understanding of experimental results requires a mechanism for motivating studies and integrating results. It requires a community of researchers that can replicate studies, vary context variables, and build models that represent the common observations about the discipline. This paper discusses the experience of the authors, based upon a collection of experiments, in terms of a framework for organizing sets of related studies. With such a framework, experiments can be viewed as part of common families of studies, rather than being isolated events. Common families of studies can contribute to important and relevant hypotheses that may not be suggested by individual experiments. A framework also facilitates building knowledge in an incremental manner through the replication of experiments within families of studies. To support the framework, this paper discusses the experiences of the authors in carrying out empirical studies, with specific emphasis on persistent problems encountered in experimental design, threats to validity, criteria for evaluation, and execution of experiments in the domain of software engineering.