Evidence-Based Software Engineering for Practitioners
IEEE Software
Journal of Systems and Software
Presenting software engineering results using structured abstracts: a randomised experiment
Empirical Software Engineering
Guidelines for conducting and reporting case study research in software engineering
Empirical Software Engineering
Future Generation Computer Systems
Developing search strategies for detecting relevant experiments
Empirical Software Engineering
A systematic review of domain analysis tools
Information and Software Technology
ESEM '09 Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement
A Taxonomy and Survey of Cloud Computing Systems
NCM '09 Proceedings of the 2009 Fifth International Joint Conference on INC, IMS and IDC
Communications of the ACM
Fundamentals of Performance Evaluation of Computer and Telecommunications Systems
Fundamentals of Performance Evaluation of Computer and Telecommunications Systems
A systematic review of comparative evidence of aspect-oriented programming
Information and Software Technology
Cloud Computing Landscape and Research Challenges Regarding Trust and Reputation
UIC-ATC '10 Proceedings of the 2010 Symposia and Workshops on Ubiquitous, Autonomic and Trusted Computing
Identifying relevant studies in software engineering
Information and Software Technology
An Empirical Investigation of Systematic Reviews in Software Engineering
ESEM '11 Proceedings of the 2011 International Symposium on Empirical Software Engineering and Measurement
Size Estimation of Cloud Migration Projects with Cloud Migration Point (CMP)
ESEM '11 Proceedings of the 2011 International Symposium on Empirical Software Engineering and Measurement
Clearing the clouds: a study of emerging scale-out workloads on modern hardware
ASPLOS XVII Proceedings of the seventeenth international conference on Architectural Support for Programming Languages and Operating Systems
How a consumer can measure elasticity for cloud platforms
ICPE '12 Proceedings of the 3rd ACM/SPEC International Conference on Performance Engineering
On searching relevant studies in software engineering
EASE'10 Proceedings of the 14th international conference on Evaluation and Assessment in Software Engineering
Towards a Taxonomy of Performance Evaluation of Commercial Cloud Services
CLOUD '12 Proceedings of the 2012 IEEE Fifth International Conference on Cloud Computing
On a Catalogue of Metrics for Evaluating Commercial Cloud Services
GRID '12 Proceedings of the 2012 ACM/IEEE 13th International Conference on Grid Computing
A factor framework for experimental design for performance evaluation of commercial cloud services
CLOUDCOM '12 Proceedings of the 2012 IEEE 4th International Conference on Cloud Computing Technology and Science (CloudCom)
Hi-index | 0.00 |
Background: Cloud Computing is increasingly booming in industry with many competing providers and services. Accordingly, evaluation of commercial Cloud services is necessary. However, the existing evaluation studies are relatively chaotic. There exists tremendous confusion and gap between practices and theory about Cloud services evaluation. Aim: To facilitate relieving the aforementioned chaos, this work aims to synthesize the existing evaluation implementations to outline the state-of-the-practice and also identify research opportunities in Cloud services evaluation. Method: Based on a conceptual evaluation model comprising six steps, the systematic literature review (SLR) method was employed to collect relevant evidence to investigate the Cloud services evaluation step by step. Results: This SLR identified 82 relevant evaluation studies. The overall data collected from these studies essentially depicts the current practical landscape of implementing Cloud services evaluation, and in turn can be reused to facilitate future evaluation work. Conclusions: Evaluation of commercial Cloud services has become a world-wide research topic. Some of the findings of this SLR identify several research gaps in the area of Cloud services evaluation (e.g., Elasticity and Security evaluation of commercial Cloud services could be a long-term challenge), while some other findings suggest the trend of applying commercial Cloud services (e.g., compared with PaaS, IaaS seems more suitable for customers and is particularly important in industry). This SLR study itself also confirms some previous experiences and records new evidence-based software engineering (EBSE) lessons.