Performance Evaluation Methodologies

team bbl


This section presents an overview of several methodologies used to evaulate software performance, particularly the Linux operating system, including tracing, workload characterization, numerical analysis, and simulation. All of these methodologies are useful in evaluating the performance of the Linux operating system. For example, much tracing and profiling occur in improving Linux performance.

Tracing

Tracing involves the instrumentation of executing applications or simulated applications to collect runtime information such as instruction and data addresses, timestamps associated with such addresses, cache misses, memory access times, register contents, and procedure or system calls. Tracing has the advantage of providing detailed runtime information that can be used to identify a performance issue. However, tracing has the disadvantage of requiring a significant amount of memory resources to store the generated trace, and the methodology for generating traces may be time-prohibitive. In addition, trace generation may occur in an obtrusive manner, changing the behavior of the executing application. Some of these disadvantages are addressed by periodically collecting trace samples. There are various methodologies for facilitating this process.

Several tools are available to help you generate traces. Typical trace tools generate log files containing timestamped records, which are post-processed to obtain such performance data as execution profiles, instruction counts, and hot spots. Linux tracing tools include strace, OProfile, the Linux Trace Toolkit (LTT), and others. More information on Linux tracing and associated tools is included in Chapter 5, "System Trace Tools."

Workload Characterization

Workload characterization involves the static or dynamic analysis of an application or workload to determine specific behaviors, such as the fraction of shared memory, or read or write accesses. The data to be analyzed may be traces generated from an instrumented application or workload, some hybrid of traces and numerical analysis, or some other source. This information can be used to provide hints regarding potential performance issues. For example, if the footprint of an application fits into the cache, but it exhibits long execution times, it may be the result of a large percentage of shared writes. Workload characterization can provide detailed information about an application or workload, which may be used to improve the design of applications, hardware, and software. However, the time required to generate this information may be significant, especially when detailed information is obtained. In addition, workload characterization for performance tuning is one of several factors used to isolate performance issues.

Numerical Analysis

Numerical analysis involves the creation of mathematical models for hardware or software systems. This methodology has the advantage of quickly obtaining data. Numerical models are useful for determining high-level trends about the behavior of systems that are simple to moderate in levels of complexity. However, the accuracy of numerical models is diminished as the level of complexity of the system modeled is increased. In addition, these models require workload characterization information, statistical random number input parameters, or other information to drive them.

Simulation

Simulation is a programmatic model of a system, and it varies from simple to complex. A simulator is typically driven by traces, events, or some other means. Simulators, specifically those that are detailed models of the system under study, offer the advantage of accuracy of results. However, they are time-consuming to execute, with some simulators executing for days, weeks, or months before producing results. In addition, simulators may also consume an extensive amount of resources. There are mechanisms for mitigating the impact of complex simulators, such as checkpointing interim results as the simulator executes. However, simulators are typically not the methodology used for tuning the performance of systems in a timely manner.

    team bbl



    Performance Tuning for Linux Servers
    Performance Tuning for Linux Servers
    ISBN: 0137136285
    EAN: 2147483647
    Year: 2006
    Pages: 254

    flylib.com © 2008-2017.
    If you may any questions please contact us: flylib@qtcs.net