Web Server Benchmarks

team bbl


This section discusses the following web server benchmarks:

  • SPECweb, SPECweb SSL, and TPC-W

  • SPECjAppServer and ECPerf

SPECweb, SPECweb SSL, and TPC-W

Web serving is a very common workload today. Everyone is interested in ensuring that their web server is fast enough to serve up whatever content they have to publish. Of course, different people publish or make available different information or services via the web. Three benchmarks worth highlighting are SPECweb, SPECweb SSL, and TPC-W. Each of these benchmarks covers a somewhat different class of web servers, although there are some overlaps. Most web serving today has a mix of static content (for example, "publish once-read often" content) and dynamic content (for example, where each page is created on-the-fly with localized or customized content). Static content is often items like news stories, PDF documents, results files, source archives, audio or video files, and so on. These documents vary in size, depending on the goals of the web server, so sometimes the static content is very small, and sometimes very large, with every possible mix in between. More often, today's coolest sites are those that allow you to log in, set your preferences, and get content tailored to your interests. This might include customized news stories, targeted advertising, personalized shopping information, and so on. SPECweb is a test designed to model the access to a web server and benchmark the rate at which the web server can provide static and dynamic content to the user. The rate of static to dynamic content is pre-set by the benchmark rules, although with some tinkering, those numbers can be adjusted to better model your own workload.

SPECweb SSL is built on top of the same SPECweb harness, but its goal is to measure the performance of encrypted data transfers using the standard Secure Sockets Layer (SSL) encryption methods. Because of the overhead of encryption, the computational power required for SSL transfers is higher on the server. Therefore the maximum number of connections that can be simultaneously supported with reasonable latency is typically less than that of non-SSL-based connections.

The third major benchmark in this class is TPC-W, which is a more comprehensive test in that it attempts to model an entire e-commerce site. This includes a mix of SSL and non-SSL accesses, includes the assumption of a database underlying the related web-based queries, and allows for transactions on that database via the web interface. In part, TPC-W is a database benchmark as well as a web-serving benchmark.

This increasing complexity in the benchmark is in some ways a method for determining if transactions are truly the sum of their components (for example, web serving, database access, file system access, processor speed, system call overhead, and so on) or if other factors are at work that increase latency or decrease throughput. In general, these complex workloads are not as fast as the microbenchmarks or more focused subsystem benchmarks might indicate. These complex workloads often have interactions that are hard to predict. In some cases, these complex workloads are run on a single machine (for example, database and web server on the same machine), where the two workloads may interact with each other. For instance, if the database consumes as much memory as the system has to offer, the web server may be constrained in how many pages it can serve up simultaneously. Or, if the database has high latency to fetch content for dynamically generated web pages, the web server may not be able to send complete web pages fast enough to keep a person browsing the web site interested.

This is where the use of a more complex benchmark, possibly in combination with some of the system monitoring tools we discussed earlier, can be extremely helpful in identifying bottlenecks and reconfiguring or optimizing the system to improve overall latencies. One solution sometimes used to improve performance is to isolate the applications that make up the various tiers of this workload, such as running the database on one system and the web server on another system. In this case, the two applications communicate over an internal network, and the results are served to the Internet via the web server. In these multitier configurations, where each tier is run on a distinct platform, the bottlenecks often shift to being network-centric. In that case, some of the networking benchmarks we mentioned earlier can help set some of the expectations for the rate at which the applications can share data on the internal network. In this case, the tuning may be very different from the environment where multiple tiers are run on the same platform.

SPECjAppServer and ECPerf

Continuing the trend in increasing complexity, SPECjAppServer (http://www.spec.org) and ECPerf (http://java.sun.com/j2ee/ecperf/index.jsp) add Java-based processing to the web serving and database components. This tends to model many of the e-commerce and general web-serving environments available on the Internet today. In these two benchmarks, the web client interacts with the web server. The web server requests content from the database, which is typically preprocessed by Java or J2EE applications before being handed to the web server for submission to the web client. In these environments, we now have a complex balance of a database (heavily I/O-centric, some CPU processing power, with heavy memory consumption) interacting with a Java or J2EE application (primarily CPU-intensive, sometimes higher memory requirements) and a web server (somewhat CPU-intensive, often high memory requirements, and typically network-intensive), all interacting via one or more internal networks. In fact, for security, quite often the database tier interacts with only the Java/J2EE machines on a private network, and the Java/J2EE machines interact with the web servers on another, also internal, network. The web servers then have full Internet access but serve as a level of protection for the more business-critical database server.

SPECjAppServer and ECPerf allow a performance analyst to configure and measure the throughput and latency for these configurations; they also allow two or more tiers to run on the same machine. In some cases, security concerns drive a three-tier configuration to run on three separate machines; in other cases, performance may drive multiple tiers to run on the same machine. In any case, the ability to benchmark the various configurations and test various databases, web servers, Java or J2EE implementations, various file systems, network topologies, CPU types, CPU speeds, memory configurations, operating or application tuning parameters, and so on allows the performance analyst to analyze various "what-if" scenarios and hopefully optimize a workload to get the best possible performance for the end user.

Other Application Benchmarks

Two other common application benchmarks worth mentioning are the Oracle Applications Standard Benchmark (http://www.oracle.com/apps_benchmark/index.html) and the SAP Standard Application Benchmark (http://www50.sap.com/benchmark/). These two benchmarks are available from the vendors of the respective products (Oracle, SAP) as a way to test their applications on various vendor platforms and operating systems. Many other vendors have benchmarks to help compare their applications against their competitors' as well as to help a performance analyst improve a local deployment of the vendor's applications. Although the performance analyst may use these tools directly, quite often the intent of these benchmarks is to allow vendors to publish numbers, which helps an analyst decide what hardware or software environment to purchase. However, these tools can also be used to analyze an environment for general configuration changes and tuning possibilities.

    team bbl



    Performance Tuning for Linux Servers
    Performance Tuning for Linux Servers
    ISBN: 0137136285
    EAN: 2147483647
    Year: 2006
    Pages: 254

    flylib.com © 2008-2017.
    If you may any questions please contact us: flylib@qtcs.net