Preparing for Load Testing

You need to make several decisions when setting up and configuring Tomcat that will affect the scalability of your installation.

Configuring Java’s Memory

The JVM sets its own memory usage, but you can configure the limits that it uses at the command line. These settings alter the JVM’s heap, which is where object instances are stored.

You should remember two very important switches when you set up a Tomcat instance.

• -Xmx<size>: The maximum heap size for the JVM

• -Xms<size>: The initial heap size for the JVM

If you don’t explicitly set these parameters, the JVM will use its defaults, which are a minimum of 2MB and a maximum of 64MB for JDK 1.4 and JDK 5.

Setting the Maximum Heap Size

Maximum heap size is the upper limit of RAM that the JVM will allocate to the heap. To set the maximum heap size to 256MB, use the following switch:

 -Xmx256m 

To specify memory size in GB, use the letter g instead of m.

In a data-intensive application with long-lived objects, memory usage can build up quickly. If an application’s memory requirement exceeds the maximum heap size of the JVM, the JVM will first default to virtual memory and then fail with and throw a java.lang.OutOfMemory error. This gives you systemwide protection, but you must be careful when setting maximum heap size. You should always make sure your system has enough memory to comply with the JVM setting because you shouldn’t rely on the virtual memory mechanism to save your server.

Setting the Minimum Heap Size

You use the initial heap size setting to allocate memory to the Java heap at JVM startup. In a memory-intensive, heavily loaded application, initial heap size can be important; if the JVM starts with a very small heap size and it receives a large number requests that require large object instantiations, it will struggle to keep up with the memory allocation needs and may not recover in some situations. In this case it’s often useful to set the minimum heap size to be the same as the maximum heap size. This will ensure there isn’t a performance hit from a large number of object instantiations at once. For example, the following sets the minimum and maximum heap sizes to 256Mb:

 -Xms256m -Xmx256m 

Something to bear in mind when doing this is that setting the heap size to a value that’s as large as your server will allow isn’t always a good idea. This may cause otherwise unexplainable pauses in the applications running on the server. It could also cause poor average server performance. Both of these phenomena are caused by the garbage collector, which runs only when memory is exhausted and then runs through the entire system. If your server handles heavy-duty applications and has a large heap, then the garbage collector has more work to do.

One possible solution to this problem is to pass the following command-line option to the Java executable:

 -Xincgc 

This forces the garbage collector to run in incremental mode, meaning it runs more often but checks through smaller amounts of memory. You should monitor this carefully because there may be a small performance hit with this method too.

Lowering the size of the heap may also help this situation, as would a combination of both these techniques. These are prime examples of why you should load test your server before it goes into production. Otherwise, you wouldn’t know which of these settings was most appropriate for the Web applications on your server.

Configuring Tomcat’s Connectors

Several connector parameters may affect your server’s performance. The following are the performance-critical attributes of the <Connector> element. For an exhaustive discussion of these elements, see Chapter 9.

The acceptCount Attribute

acceptCount sets the number of connections that the server will accept while waiting for a free processor. Incoming connections over this limit will be refused. While you may be tempted to increase this to a very high number, a high setting may cause your system to run out of free file descriptors, which can cause processes—or under extreme circumstances, operating systems— to crash or become unstable. The default is 10.

The enableLookups Attribute

This setting tells Tomcat to resolve each request’s host name. This is useful when viewing log files, but it puts extra load on the server and the network. You should therefore use it with caution. The default is true.

The maxProcessors Attribute

The maxProcessors attribute imposes a limit on the number of threads the server will start, regardless of the server load. If the server receives more simultaneous requests for a given connection than the value of this setting, the requests will block until a thread is freed to handle them. If this number is set too high, heavily loaded sites run the risk of a performance slowdown as the JVM struggles to manage the large number of threads and network connections that will be created.

You can monitor thread count with operating system–specific tools, such as ps in Unix-like systems. If the number of threads approaches the maxProcessors setting, followed by a server performance slowdown, you should increase this setting and repeat the experiment. The default is 20.

The minProcessors Attribute

A processor is a thread that handles requests for a connector on a given port. Setting the minProcessors attribute too high can produce a large number of unnecessary threads, which will put an extra burden on the JVM. Setting it too low can cause delays when servicing requests that come in soon after server startup because the server will have to spawn a separate thread for incoming requests if it’s already servicing the number of clients equal to this setting.

As with the maxProcessors attribute, you can monitor the thread count with operating system–specific tools. If you see the number of threads increasing rapidly before reaching a plateau, the number of threads reached at the plateau makes a good general minProcessors setting. The default is 5.

Configuring Application Sessions

Tomcat’s default session manager is very fast because it stores its data in memory, as discussed in Chapter 7. This implies a trade-off between speed and the memory consumption on the server. However, the problem when working with sessions is that they’re configured at the application level in web.xml, using the <session-timeout> subelement of the <session-config> element. This means developers are in charge of them in the beginning and may have their own reasons for configuring them they way they are.

You must weigh the needs of your server against the needs of the developer’s application and its users. Ultimately, you have responsibility for the application once it’s deployed on your server, so you have the means and the authority to change the session settings as appropriate.

In extreme cases, such as data-entry applications or point-of-sale systems, where sessions need to be active for hours at a time, it may be worthwhile to use Tomcat’s persistent session manager. Reactivation of the sessions will be sluggish in terms of performance, but the memory trade-off may prove to be worth the cost. Chapter 7 also covered the persistent session manager.

Altering Tomcat’s Deployment Architecture

The simplest Tomcat setup with a single stand-alone Tomcat server using an HTTP connector is usually appropriate for very small installations. However, as load increases and applications become resource intensive, the deployment architecture can make or break a server’s performance.

It’s possible, under certain conditions, for the JVM to become a bottleneck, even if a single server is sufficient. The JVM isn’t optimized for dealing with huge amounts of memory, so breaking it into multiple processes on the same system may help, as discussed in Chapter 13.

If application performance is constrained by the limits of the operating system or server hardware, it may be necessary to load balance two or more application servers, as discussed in Chapter 9.

While Tomcat has an HTTP connector, it isn’t optimized as an HTTP server. Bringing Apache or other supported Web servers into the picture would increase performance, as they’re designed for handling only HTTP requests, as discussed in Chapter 9.

Working with a Developer’s Code

A well-configured server is no match for inefficient application code deployed within it. The best weapon in this situation is a clear understanding of the performance of your server when it’s unencumbered with sluggish code. Regardless of what the reality is, the onus is always on you as the server administrator to identify the bottleneck. Thorough pre-application load testing and analysis will allow you to cast off undeserved blame and quickly identify application performance bottlenecks as and when they appear.