Using concept mapping for maintainability assessments
ESEM '09 Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement
The usability inspection performance of work-domain experts: An empirical study
Interacting with Computers
The challenge of assessing and controlling complexity in a large portfolio of software systems
Proceedings of the 11th International Conference on Product Focused Software
Questioning software maintenance metrics: a comparative case study
Proceedings of the ACM-IEEE international symposium on Empirical software engineering and measurement
Plat_Forms 2011: finding emergent properties of web application development platforms
Proceedings of the ACM-IEEE international symposium on Empirical software engineering and measurement
Exploring the impact of inter-smell relations on software maintainability: an empirical study
Proceedings of the 2013 International Conference on Software Engineering
To what extent can maintenance problems be predicted by code smell detection? - An empirical study
Information and Software Technology
| Hi-index | 0.00 |
The scientific study of a phenomenon requires it to be reproducible. Mature engineering industries are recognized by projects and products that are, to some extent, reproducible. Yet, reproducibility in software engineering (SE) has not been investigated thoroughly, despite the fact that lack of reproducibility has both practical and scientific consequences. We report a longitudinal multiple-case study of variations and reproducibility in software development, from bidding to deployment, on the basis of the same requirement specification. In a call for tender to 81 companies, 35 responded. Four of them developed the system independently. The firm price, planned schedule, and planned development process, had, respectively, “low,” “low,” and “medium” reproducibilities. The contractor's costs, actual lead time, and schedule overrun of the projects had, respectively, “medium,” “high,” and “low” reproducibilities. The quality dimensions of the delivered products, reliability, usability, and maintainability had, respectively, “low,” "high,” and “low” reproducibilities. Moreover, variability for predictable reasons is also included in the notion of reproducibility. We found that the observed outcome of the four development projects matched our expectations, which were formulated partially on the basis of SE folklore. Nevertheless, achieving more reproducibility in SE remains a great challenge for SE research, education, and industry.