Position paper: cloud-based performance testing: issues and challenges

Authors:
Junzan Zhou;Shanping Li;Zhen Zhang;Zhen Ye
Affiliations:
Zhejiang University, Hangzhou, China;Zhejiang University, Hangzhou, China;Zhejiang University, Hangzhou, China;Zhejiang University, Hangzhou, China
Venue:
Proceedings of the 2013 international workshop on Hot topics in cloud services
Year:
2013

Citing 21
Cited 0

The NIST model for role-based access control: towards a unified standard

RBAC '00 Proceedings of the fifth ACM workshop on Role-based access control
Integrated Design of AES (Advanced Encryption Standard) Encrypter and Decrypter

ASAP '02 Proceedings of the IEEE International Conference on Application-Specific Systems, Architectures, and Processors
SLAng: A Language for Defining Service Level Agreements

FTDCS '03 Proceedings of the The Ninth IEEE Workshop on Future Trends of Distributed Computing Systems
Performance by Design: Computer Capacity Planning By Example

Performance by Design: Computer Capacity Planning By Example
RBSLA A declarative Rule-based Service Level Agreement Language based on RuleML

CIMCA '05 Proceedings of the International Conference on Computational Intelligence for Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce Vol-2 (CIMCA-IAWTIC'06) - Volume 02
Performance Modeling and Evaluation of Distributed Component-Based Systems Using Queueing Petri Nets

IEEE Transactions on Software Engineering
A provisioning model and its comparison with best-effort for performance-cost optimization in grids

Proceedings of the 16th international symposium on High performance distributed computing
CPU demand for web serving: Measurement analysis and dynamic estimation

Performance Evaluation
A regression-based analytic model for capacity planning of multi-tier applications

Cluster Computing
The cost of doing science on the cloud: the Montage example

Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Empirical evaluation of latency-sensitive application performance in the cloud

MMSys '10 Proceedings of the first annual ACM SIGMM conference on Multimedia systems
Software Testing as an Online Service: Observations from Practice

ICSTW '10 Proceedings of the 2010 Third International Conference on Software Testing, Verification, and Validation Workshops
Large-Scale Software Testing Environment Using Cloud Computing Technology for Dependable Parallel and Distributed Systems

ICSTW '10 Proceedings of the 2010 Third International Conference on Software Testing, Verification, and Validation Workshops
Migrating Autonomic Self-Testing to the Cloud

ICSTW '10 Proceedings of the 2010 Third International Conference on Software Testing, Verification, and Validation Workshops
Automated software testing as a service

Proceedings of the 1st ACM symposium on Cloud computing
Adoption issues for cloud computing

Proceedings of the 7th International Conference on Advances in Mobile Computing and Multimedia
Service Level Checking in the Cloud Computing Context

CLOUD '10 Proceedings of the 2010 IEEE 3rd International Conference on Cloud Computing
Runtime measurements in the cloud: observing, analyzing, and reducing variance

Proceedings of the VLDB Endowment
Software Testing Based on Cloud Computing

ICICIS '11 Proceedings of the 2011 International Conference on Internet Computing and Information Services
Towards Automatic Performance and Scalability Testing of Rich Internet Applications in the Cloud

SEAA '11 Proceedings of the 2011 37th EUROMICRO Conference on Software Engineering and Advanced Applications
A security analysis of amazon's elastic compute cloud service

Proceedings of the 27th Annual ACM Symposium on Applied Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Conducting performance testing is essential to evaluate system performance. With the emergence of cloud computing, applying cloud resources for large-scale performance testing become very attractive. Many organizations have applied cloud-based performance testing in realistic projects. Cloud computing brings many benefits for performance testing, while we also have to face many new problems such as performance variation of cloud platform and security problems. In this overview, we discuss the differences between traditional and cloud-based performance testing. We investigate the state-of-art of cloud-based performance testing. We address the key issues with relevant challenges. For some of the issues, we formalize the problems and give our initial idea. We focus on the quality of workload generation and present our experimental results to validate the existence and degree of the challenges. We think that it is beneficial to apply cloud-based performance testing in many cases.