4.2 Performance Engineering

It has been a common practice in many systems projects to consider performance requirements only at the final stages of the software development process. As a consequence, many systems exhibit performance failures that lead to delays and financial losses to companies and users [25]. Performance engineering analyzes the expected performance characteristics of a system during the different phases of its lifecycle. Performance engineering 1) develops practical strategies that help predict the level of performance a system can achieve and 2) provides recommendations to realize the optimal performance level. Both tasks rely on the following activities that form the basis of a methodology.

• Understand the key factors that affect a system's performance.

• Measure the system and understand its workload.

• Develop and validate a workload model that captures the key characteristics of the actual workload.

• Develop and validate an analytical model that accurately predicts the performance of the system.

• Use the models to predict and optimize the performance of the system.

Performance engineering can be viewed as a collection of methods for the support of the development of performance-oriented systems throughout the entire lifecycle [7]. The phases of the system lifecycle define the workflow, procedures, actions, and techniques that are used by analysts to produce and maintain IT systems. Figure 4.1 provides an overview of a performance-based methodology for system design and analysis. This methodology is based on models that are used to provide QoS assurances to IT systems. These models are: workload model, performance model, availability model, reliability model, and cost model. At the early stages of a project, the information and data available to develop the models are approximate at best.

Figure 4.1. Modeling process.

For instance, in the requirement analysis stage, it is difficult to construct a detailed workload model, specifying the demands placed by transactions on the system's resources. However, as the project evolves, more system information becomes available, reducing the error margin of the model's assumptions and increasing confidence in the estimated predictions of the future system's performance [23]. In summary, the workload and performance models evolve side by side with the system project throughout the stages of its lifecycle. Various types of workload and performance models can be used during the system lifecycle. Basically, they differ in three aspects: complexity, accuracy, and cost. Figure 4.1 illustrates the spectrum of the models for the different phases, ranked from least to most complex and costly. It is expected that as complexity and cost increase, the refined models becomes more accurate. At the low end of the figure, one may use back-of-the-envelope calculations as a simple, easy-to-use technique, with a relatively low level of accuracy. At the high end, detailed models of running systems are used that are accurate but, are also costly and complex.