Proceedings of the 2008 ACM SIGMOD international conference on Management of data
Amazon S3 for science grids: a viable solution?
DADC '08 Proceedings of the 2008 international workshop on Data-aware distributed computing
The cost of doing science on the cloud: the Montage example
Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Proceedings of the 2008 annual research conference of the South African Institute of Computer Scientists and Information Technologists on IT research in developing countries: riding the wave of technology
Future Generation Computer Systems
Evaluating the cost-benefit of using cloud computing to extend the capacity of clusters
Proceedings of the 18th ACM international symposium on High performance distributed computing
C-Meter: A Framework for Performance Analysis of Computing Clouds
CCGRID '09 Proceedings of the 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid
Cost-benefit analysis of Cloud Computing versus desktop grids
IPDPS '09 Proceedings of the 2009 IEEE International Symposium on Parallel&Distributed Processing
How is the weather tomorrow?: towards a benchmark for the cloud
Proceedings of the Second International Workshop on Testing Database Systems
Performance Evaluation of Cloud Computing Offerings
ADVCOMP '09 Proceedings of the 2009 Third International Conference on Advanced Engineering Computing and Applications in Sciences
Web Server Farm in the Cloud: Performance Evaluation and Dynamic Architecture
CloudCom '09 Proceedings of the 1st International Conference on Cloud Computing
Communications of the ACM
Benchmarking cloud serving systems with YCSB
Proceedings of the 1st ACM symposium on Cloud computing
An evaluation of alternative architectures for transaction processing in the cloud
Proceedings of the 2010 ACM SIGMOD International Conference on Management of data
Fundamentals of Performance Evaluation of Computer and Telecommunications Systems
Fundamentals of Performance Evaluation of Computer and Telecommunications Systems
WAINA '10 Proceedings of the 2010 IEEE 24th International Conference on Advanced Information Networking and Applications Workshops
The impact of virtualization on network performance of amazon EC2 data center
INFOCOM'10 Proceedings of the 29th conference on Information communications
Early observations on the performance of Windows Azure
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
Case study for running HPC applications in public clouds
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
AzureBlast: a case study of developing science applications on the cloud
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
CloudCmp: comparing public cloud providers
IMC '10 Proceedings of the 10th ACM SIGCOMM conference on Internet measurement
Performance measurement of a private cloud in the OpenCirrus™ testbed
Euro-Par'09 Proceedings of the 2009 international conference on Parallel processing
Runtime measurements in the cloud: observing, analyzing, and reducing variance
Proceedings of the VLDB Endowment
EC2 performance analysis for resource provisioning of service-oriented applications
ICSOC/ServiceWave'09 Proceedings of the 2009 international conference on Service-oriented computing
Abstractions for Loosely-Coupled and Ensemble-Based Simulations on Azure
CLOUDCOM '10 Proceedings of the 2010 IEEE Second International Conference on Cloud Computing Technology and Science
Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud
CLOUDCOM '10 Proceedings of the 2010 IEEE Second International Conference on Cloud Computing Technology and Science
Cost-Effective HPC: The Community or the Cloud?
CLOUDCOM '10 Proceedings of the 2010 IEEE Second International Conference on Cloud Computing Technology and Science
Evaluating Cloud Platform Architecture with the CARE Framework
APSEC '10 Proceedings of the 2010 Asia Pacific Software Engineering Conference
Metabolic Flux Analysis in the Cloud
ESCIENCE '10 Proceedings of the 2010 IEEE Sixth International Conference on e-Science
An Empirical Investigation of Systematic Reviews in Software Engineering
ESEM '11 Proceedings of the 2011 International Symposium on Empirical Software Engineering and Measurement
How a consumer can measure elasticity for cloud platforms
ICPE '12 Proceedings of the 3rd ACM/SPEC International Conference on Performance Engineering
Scientific Computing in the Cloud
Computing in Science and Engineering
Towards a Taxonomy of Performance Evaluation of Commercial Cloud Services
CLOUD '12 Proceedings of the 2012 IEEE Fifth International Conference on Cloud Computing
On evaluating commercial Cloud services: A systematic review
Journal of Systems and Software
| Hi-index | 0.00 |
Given the continually increasing amount of commercial Cloud services in the market, evaluation of different services plays a significant role in cost-benefit analysis or decision making for choosing Cloud Computing. In particular, employing suitable metrics is essential in evaluation implementations. However, to the best of our knowledge, there is not any systematic discussion about metrics for evaluating Cloud services. By using the method of Systematic Literature Review (SLR), we have collected the de facto metrics adopted in the existing Cloud services evaluation work. The collected metrics were arranged following different Cloud service features to be evaluated, which essentially constructed an evaluation metrics catalogue, as shown in this paper. This metrics catalogue can be used to facilitate the future practice and research in the area of Cloud services evaluation. Moreover, considering metrics selection is a prerequisite of benchmark selection in evaluation implementations, this work also supplements the existing research in benchmarking the commercial Cloud services.