Automatic detection of performance deviations in the load testing of large scale systems
Proceedings of the 2013 International Conference on Software Engineering
Performance optimization of deployed software-as-a-service applications
Journal of Systems and Software
Continuous validation of load test suites
Proceedings of the 5th ACM/SPEC international conference on Performance engineering
| Hi-index | 0.00 |
Load testing is crucial to uncover functional and performance bugs in large-scale systems. Load tests generate vast amounts of performance data, which needs to be compared and analyzed in limited time across tests. This helps performance analysts to understand the resource usage of an application and to find out if an application is meeting its performance goals. The biggest challenge for performance analysts is to identify the few important performance counters in the highly redundant performance data. In this paper, we employed a statistical technique, Principal Component Analysis (PCA) to reduce the large volume of performance counter data, to a smaller, more meaningful and manageable set. Furthermore, our methodology automates the process of comparing the important counters across load tests to identify performance gains/losses. A case study on load test data of a large enterprise application shows that our methodology can effectively guide performance analysts to identify and compare top performance counters across tests in limited time.