Independent verification & validation: a life cycle engineering process for quality software
Independent verification & validation: a life cycle engineering process for quality software
Test Case Prioritization: An Empirical Study
ICSM '99 Proceedings of the IEEE International Conference on Software Maintenance
Synthesizing SoS concepts for use in cost modeling
Systems Engineering
Guidelines for conducting and reporting case study research in software engineering
Empirical Software Engineering
Context in industrial software engineering research
ESEM '09 Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement
Perceived productivity threats in large agile development projects
Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement
Service Orientation and Systems of Systems
IEEE Software
A Study of the Characteristics of Behaviour Driven Development
SEAA '11 Proceedings of the 2011 37th EUROMICRO Conference on Software Engineering and Advanced Applications
Refactoring and metrics for TTCN-3 test suites
SAM'06 Proceedings of the 5th international conference on System Analysis and Modeling: language Profiles
Regression testing minimization, selection and prioritization: a survey
Software Testing, Verification & Reliability
Hi-index | 0.00 |
Context: Systems of systems (SoS) are highly complex and are integrated on multiple levels (unit, component, system, system of systems). Many of the characteristics of SoS (such as operational and managerial independence, integration of system into system of systems, SoS comprised of complex systems) make their development and testing challenging. Contribution: This paper provides an understanding of SoS testing in large-scale industry settings with respect to challenges and how to address them. Method: The research method used is case study research. As data collection methods we used interviews, documentation, and fault slippage data. Results: We identified challenges related to SoS with respect to fault slippage, test turn-around time, and test maintainability. We also classified the testing challenges to general testing challenges, challenges amplified by SoS, and challenges that are SoS specific. Interestingly, the interviewees agreed on the challenges, even though we sampled them with diversity in mind, which meant that the number of interviews conducted was sufficient to answer our research questions. We also identified solution proposals to the challenges that were categorized under four classes of developer quality assurance, function test, testing in all levels, and requirements engineering and communication. Conclusion: We conclude that although over half of the challenges we identified can be categorized as general testing challenges still SoS systems have their unique and amplified challenges stemming from SoS characteristics. Furthermore, it was found that interviews and fault slippage data indicated that different areas in the software process should be improved, which indicates that using only one of these methods would have led to an incomplete picture of the challenges in the case company.