Use case-driven performance engineering without "concurrent users"

• Authors: Morten Heine Sørensen
• Venue: Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering
• Year: 2013

The concept of concurrent users often causes confusion when used to define performance requirements in industrial software projects. The term is frequently used to state performance requirements without clarification of what the users will be doing, or how often. This paper offers a thorough analysis of the concept and related notions. Despite the confusion surrounding it, the concept of concurrent users -- in a precise form -- is advocated in the community for stating performance requirements. However, we argue in this paper that, even when stated in precise terms, this approach has drawbacks. Indeed, a system may perform better than expected, even if the number of concurrent users it can handle is worse than expected. A better suited notion is that of through-put. But even when basing performance requirements on clear, well-suited concepts, there appears to be no uniform format in the literature for such requirements. In particular, the requirements are sometimes stated in general, rather than for the specific areas of functionality of the system. As a consequence, the point may be missed that the through-put may be unevenly distributed over the functionality of the system. In this paper we therefore advocate the format of performance-annotated use cases, adding requirements on through-put and response-time to the traditional use case. It is well-known how functional test cases are developed from use cases. In contrast, less has been said about the generation of performance test cases. Therefore, we show how the enriched use cases not only provide precise and meaningful requirements, but also yield detailed specification of the performance test set-up which can be directly input as configuration of load test clients. As a bonus, initial configuration of the system's capacity for handling concurrent users and requests is also provided. Finally we outline an overall approach to performance test based on the above ideas. The approach has been followed in several industrial projects.