People, Organizations, and Process Improvement
IEEE Software
Using benchmarking to advance research: a challenge to software engineering
Proceedings of the 25th International Conference on Software Engineering
Empirical Software Engineering
The DaCapo benchmarks: java benchmarking development and analysis
Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications
FLOSSMetrics: Free/Libre/Open Source Software Metrics
CSMR '09 Proceedings of the 2009 European Conference on Software Maintenance and Reengineering
Benefits of global software development: the known and unknown
ICSP'08 Proceedings of the Software process, 2008 international conference on Making globally distributed software development a success story
Experiences mining open source release histories
Proceedings of the 2011 International Conference on Software and Systems Process
Release engineering practices and pitfalls
Proceedings of the 34th International Conference on Software Engineering
Concept location using program dependencies and information retrieval (DepIR)
Information and Software Technology
Distributed development considered harmful?
Proceedings of the 2013 International Conference on Software Engineering
Hi-index | 0.00 |
Empirical studies that use software repository artifacts have become popular in the last decade due to the ready availability of open source project archives. In this paper, we survey empirical studies in the last three years of ICSE and FSE proceedings, and categorize these studies in terms of open source projects vs. proprietary source projects and the diversity of subject programs used in these studies. Our survey has shown that almost half (49%) of recent empirical studies used solely open source projects. Existing studies either draw general conclusions from these results or explicitly disclaim any conclusions that can extend beyond specific subject software. We conclude that researchers in empirical software engineering must consider the external validity concerns that arise from using only several well-known open source software projects, and that discussion of data source selection is an important discussion topic in software engineering research. Furthermore, we propose a community research infrastructure for software repository benchmarks and sharing the empirical analysis results, in order to address external validity concerns and to raise the bar for empirical software engineering research that analyzes software artifacts.