Tuning the Java Virtual Machine

All Java applications rely on a Java Virtual Machine (JVM) to execute and making sure your JVM is running as efficiently as possible is important to make sure that your ColdFusion MX 7 applications are working as efficiently as possible as well. In the next sections we are going to look at how the JVM works, how to Garbage Collection works, how to change and control the JVM, and how to monitor the JVM via log files and by visualizing the JVM so that you can tune the JVM for optimal performance.

Introducing the JVM

A JVM is an implementation of Sun's Java Virtual Machine specification, which defines an abstract machine or processor that is basically a software version of a hardware processor or CPU. The JVM spec also specifies an instruction set, a set of registers, a stack, a "garbage heap," and what is called a method area. Software companies (including Microsoft and IBM), vendors, and others create JVMs for specific platforms, although by far the most popular is Sun's JVM. Once a JVM has been created for a given platform, any Java program (compiled Java source, or bytecode) can run on that platform. A JVM can either interpret bytecode one instruction at a time, mapping it to a real processor instruction; or it can further compile the bytecode for the real processor, using what is called a just in time (JIT) compiler, which radically improves performance.

NOTE

For a more detailed look at the Sun JVM specification, look at http://java.sun.com/docs/books/vmspec/.

The JVM is the heart of any Java application and so must work as efficiently as possible. One of the main reasons a JVM may perform poorly is that it's not using memory efficiently. In general, the JVM is very efficient, but it can use some help in high-performance applications where it's expected to do a lot of work, manage a large number of objects, deal with large data sets, or simply operate as fast as possible. In this section we focus on some things you can do make your JVM excel for your specific application.

ColdFusion MX 7 ships with a series of default JVM attributes that are designed to work on almost any system or setup but are not the best configuration for every application. This is also true for almost every other J2EE application server, and knowledgeable J2EE system managers routinely profile and tune their J2EE servers based on their particular system and application operation parameters. When tuning your JVM, the first place to start is to figure out what the heap value for your JVM will be. Java applications make use of a garbage collection heap, which is a specialized memory structure. "The heap" is where all the objects of a Java program live. This means every ColdFusion page, CFC, session variable, and so on that becomes a Java object resides in the heap.

The JVM periodically performs "garbage collection" to remove or de-allocate unused Java objects from the heap. Your heap value and settings determine in part how often and for how long garbage collection occurs. These settings are very important because, while garbage collection is happening, the JVM essentially pauses and puts all other operations and requests on hold. Usually garbage collection takes so little time and is so efficient that it has relatively no impact on performance, but in certain instances where there are few resources or the system is under high load, performance can take a hit during garbage collection.

If the heap value for your JVM is too low, you may see such things as the "Java.lang.OutOfMemory" error, ColdFusion server timeouts, poor response times under load, and what appear to be memory leaks. Furthermore, garbage collection may occur so often that it affects performance, which also leads to similar issues. Numerous other things unrelated to JVM tuning can cause these behaviors, but often the reason is that the application is demanding more from the JVM than it can deliver with resources it has.

ColdFusion's default setting for the size of the JVM heap is 512 MB. (The JVM settings, jvm.config, are in server_root\bin.) This is often sufficient for most applications, but if your application has a large amount of traffic, makes lots of large database queries, or otherwise requires more heap space, then you will want to set it higher. The easiest way to tell what your heap should be is by profiling your application using a product such as OptimizeIT (http://www.borland.com/optimizeit/) or JProfile (http://www.ej-technologies.com/). These programs help you easily determine the amount of memory your application is using and how it is using it. Another method is to use a performance monitoring tool, such as perfmon on Windows, and monitor the application under load. This way you can get a sense of memory usage even at peak times. If you find that the system is often using all the memory available to it, then you'll want to increase what you allocate to the heap.

Generally, if your server has substantial free memory, you'll want to assign it to the heap. A server with 2 GB memory can afford a JVM heap maximum setting of 1000 MB. A good rule is to assign the JVM as much free memory as possible without causing the system to swap pages to disk. On production systems, you should also set the JVM heap's minimum to be the same as the maximum. This reduces periodic pauses of the JVM as it allocates more memory to itself, up to the maximum setting, and reduces what is called heap fragmentation.

So to set the maximum and minimum heap to 1 GB, you'd edit the jvm.config for each server instance that is running ColdFusion. On your J2EE server, look for # Arguments to VM and edit:

 java.args=-server -Xmx512m 

to read:

 java.args=-server -Xmx1000m -Xms1000m 

You don't want to touch the text java.args=-server, which is what Java expects to see to let it know that everything after that attribute comprises settings for the JVM. The -Xmx stands for heap maximum, and the -Xms stands for heap minimum.

TIP

Since the JVM is memory intensive, and memory is one of the largest bottlenecks for any Java application, always try to reasonably maximize memory on your servers. Memory is cheap, and having at least a gigabyte of memory available to your JVM heap will make a significant positive impact on performance.

NOTE

JVM tuning may not be rocket science, but it's not trivial, either. If you want more information about tuning Sun's JVM 1.4+, check out http://java.sun.com/docs/hotspot/gc1.4.2/. If you use a different JVM, read the vendor's documentation. You should also read the official Sun JVM Tuning FAQ here: http://java.sun.com/docs/hotspot/gc1.4.2/faq.html.

Tuning Garbage Collection

Now that we have discussed what the JVM is and how to set the heap size, let's take a look at how to tune garbage collection. Generally the JVM does a good job of cleaning up all the unused objects in the heap, but if you have profiled your application, you can optimize the JVM for the specific application. In this section we'll describe some of the most common options and how to set them in jvm.config. Then we'll see how you can use the jvmstat tool and visualgc to help you determine what settings you will want to use for your specific application's JVM.

TIP

Incorrectly setting JVM attributes or even a simple typo can cause your J2EE server to fail to start. Simply replacing the jvm.config file with one that has the original settings will resolve this, so we suggest that you keep a copy of the original settings. Or, better yet, keep your jvm.config in a version control system with updates and notes every time you change the file.

Before we start discussing specific garbage collection options, let's take a closer look at heap memory usage and garbage collection. In this section, you'll encounter some special Java terminology that will be important not only for understanding the garbage collection but also the garbage collection log files. If you plan to read more about tuning the JVM, you'll need an understanding of these terms.

The JVM breaks up the heap into pools or buckets of memory, into which are assigned Java objects that have specific life expectancies; after these periods of time end, the objects need to be cleaned up. These memory buckets are called generations, based on the life expectancy of their occupants. There are three major generations: young, tenured, and perm (for permanent, which is actually a special type of tenured generation). Objects that reside in the young generation are typically only going to last for, say, the life cycle of a ColdFusion page request. Objects with longer life expectancy, such as a cached query or even the classes that make up ColdFusion itself, would reside in the tenured generation. The perm generation contains objects that make up the actual JVM. Figure 4.13 will give you an idea of the heap's generations.

Figure 4.13. The heap is divided into the young, tenured, and permanent generations.

The young generation is probably the most important. It's made of three segments, eden and two survivor spaces. Eden is where Java objects are initially created and assigned and, if they are not immediately cleaned up, they are assigned to one of the survivor spaces. Survivor objects that are still needed are then copied to the tenured space. The young generation setting is, after the heap size, probably the most important setting and has the greatest impact on performance. The more memory assigned to the young generation, the less minor garbage collection has to be done by the JVM. This has a potential drawback: The greater your young generation, the smaller your tenured generation can be. This means the JVM has to more often do major garbage collection, which essentially pauses the JVM while it cleans up the tenured space. Depending on your application and the amount of memory available, this is usually not a problem.

You can use several attributes to control the young generation. Probably the most important of these are NewSize and MaxNewSize, which essentially function like minimum and maximum limits. You may notice the absence of these settings in jvm.config (they are optional). If they do not exist, the JVM tries to figure out the best settings dynamically, and it usually does a decent job. However, by experimenting with these settings you can often greatly improve performance.

Setting NewSize and MaxNewSize to be identical is just like setting Xms and Xmx for the heap to reduce the JVM's workload. Generally, setting the young generation to 25 percent of the total heap works well, but you'll need to test these settings to find the best size for your particular system and application. For example, to set the young generation size to 100 MB, add these strings to the Java arguments (java.args) string:

 -XX:NewSize=100m -XX:NewMaxSize=100m 

Finally, it's important to mention the permanent generation. Some applications, including ColdFusion MX 7, dynamically create and load a hefty quantity of classes into memory. With large ColdFusion applications, this can be an issue. This is often seen in situations where the "Java.lang.OutOfMemory" error crops up in your log files and can be easily solved by increasing the maximum permanent generation (MaxPermSize). For example, to increase maximum permanent generation size to 256 MB, add this to the JVM arguments:

 -XX:MaxPermSize=256m 

Selecting a Collector

The next most important thing after deciding on the size of your young generation is determining what garbage collection method to use. Generally, the JVM's default collector is fineespecially if your server has a single CPUbut other collectors can offer performance increases in these situations, as well as on servers that have more than one CPU. There are three different types of collectors: throughput, concurrent, and incremental.

The throughput collector is set by adding the string:

 -XX:+UseParallelGC 

to the JVM arguments string. This collector is a parallel version of the default young generation collector and is especially useful on servers with many CPUs and a large young generation.

TIP

The throughput collector is especially powerful on systems with many processors because, unlike the default collector, you can have the throughput collector use multiple threads for garbage collection. You can do this by adding the string-XX:ParallelGCThreads=<desired number> after -XX:+UseParallelGC, where desired number is usually the number of CPUs on the server.

The concurrent collector is set by adding the string:

 -XX:+UseConcMarkSweepGC 

after the JVM arguments string. This one is often called the "low pause" collector, in that it pauses the application for very short periods while it does collection on the tenured generation. You can also add a parallel version of the young generation collector, by adding this combination of parameters:

 -XX:+UseConcMarkSweepGC -XX:+ParNewGC 

to the JVM arguments string. If you decide to use this option, you'll need to delete the -XX:+UseParallelGC collector; this is usually the default collector that ships with ColdFusion's jvm.config file.

Finally, there is the incremental collector, which performs many small collections on the tenured generation but in most cases is slower than the default collection setting. You can set the incremental collector by adding:

 -Xincgc 

to the JVM arguments string.

All these collectors give you alternatives to the ColdFusion's default collector, but changing the default collector is the last thing you should attemptonly after increasing your heap size and tuning your young generation. After you have made those changes, you can try changing the collector to see if any of the optional collectors will increase performance.

Testing and Tuning

Now that you understand how memory is allocated, how to assign memory to the heap and the various generations, and how to assign specific garbage collections methods to the JVM, you should test your application. The simplest way to do this is generally to use your testing suite to put the application under load and monitor the effects of the new settings on application performance. If you don't see increases in throughput, try a different approach.

Another approach, especially if you are seeing intermittent errors or performance issues, is to dump the output of garbage collection to your logs. You do this by simply adding these parameters to your JVM arguments:

 -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC 

• The -XX:+PrintGCDetails tells ColdFusion to print out garbage collection details.

• The -XX:+PrintGCTimeStamps tells it to add timestamps (which can be really useful in debugging so that you can correlate problems to specific times).

• The -XX:+PrintHeapAtGC will print out detailed information on the generations. By default, ColdFusion will output this data to serverinstance_root\runtime\logs\<servername>-out.log.

If you would like to log the garbage collection data to its own file, you can also add -Xloggc:<filename>, where filename is chosen by you.

• The output from the garbage collection will look something like this:

  [View full width]
   11948.248: [Full GC  {Heap before GC invocations=248: Heap  def new generation   total 49088K, used 4735K [0x10010000, 0x13010000, 0x13010000)   eden space 49024K,   9% used [0x10010000, 0x104afda0, 0x12ff0000)   from space 64K,   0% used [0x12ff0000, 0x12ff0000, 0x13000000)   to   space 64K,   0% used [0x13000000, 0x13000000, 0x13010000)  concurrent mark-sweep generation total 999424K, used 28683K [0x13010000, 0x50010000,  0x50010000)  concurrent-mark-sweep perm gen total 65536K, used 22628K [0x50010000, 0x54010000, 0x58010000) 

  
  

At first this might be intimidating and hard to read, but look at it carefully and it will start to make sense. Let's break it down. The first line marks specific events in seconds from when ColdFusion was started. In this example, a full garbage collection was started. Next comes detailed information about the various generations in the heap. Notice the first one, the new generation, which is the same thing as young generation; it has a total of 49088K allocated to it but is only using 4735K of the memory. The next two lines after the eden space are the from and to, which are essentially the survivor spaces; they are totally empty.

The next line shows that the concurrent garbage collection method was used, that the total heap was 999424K (1 gigabyte), and only 28683K was used. The last line shows that the concurrent collector was used on the permanent generation and that its size was 65536K, with only 22628K used.

As you can see, the output from the garbage collection is not too difficult to understand but it can be difficult to read and determine what actions you need to take just from look at a few lines in your log files. There are garbage collection log file readers such as HP's HPjtune (http://www.hp.com/products1/unix/java/java2/hpjtune/index.html), a free tool that reads garbage collection log files you have created and provides a variety of visual reports that may make your job easier. You should look into a tool like this in that you can look at log files from a week of actual usage and get a much better sense of how your system is behaving. Many of the more sophisticated commercial products also have built-in garbage collection analysis tools. If you find yourself often having to tune the JVMs of your application, then an investment in one of these will be well worth the money.

Not surprisingly, many ColdFusion developers as well as system administrators would rather not learn the JVM specification by heart. That said, knowing how the JVM functions and how to tune it will greatly increase your ability not just to troubleshoot possible errors and performance problems, but to squeeze the maximum performance from your ColdFusion applications.

Visualizing Garbage Collections

One of the hardest things about tuning the JVM is understanding the information that is being generated in your log files and trying to relate that to your settings and performance. While there are excellent tools such as HP's Jtune for understanding your JVM Garbage collection log files a more intuitive way to tune your JVM is by watching actually seeing how each of the various generations are being used by the JVM while your application is under load. You can do this by using the visualgc tool which was described in chapter 2 Monitoring System performance.

If you configure your system to use the visualgc tool and then monitor your application under load you should see something like Figure 4.14. You should see all the JVM attributes set in your jvm.config file as well as graphical information for the tenured space (called Old Gen in visualgc), permanent generation, Eden (Young Generation) and Survivor 0 and Survivor 1. You should also see graphs for all these information as well as information for Garbage Collection which gives the total number of collections and the collective amount of time garbage collection has taken.

Figure 4.14. This figure shows the JVM for a simple ColdFusion MX 7 application while being place under load by a load testing tool.

In Figure 4.14 we see that the various memory spaces are barley being used garbage collection is pretty efficient. In more extreme cases you might see the memory spaces completely full. For example in Figure 4.15 we see the same application after it has been running for a longer period of time and you can see that the Eden space is now being used far more extensively. In a situation like this you might consider increasing the young generation size so as to decrease garbage collection frequency and then rerun your load test and see how performance changes.

Figure 4.15. This figure shows the same application later as more resources are being used.

Usually you will want to let your application run under load for some period of time to really see if your JVM is having issues. Often you will see in the visualgc tools that your young or tenured generation space is being utilized heavily and then may see a corresponding increase in both the number of garbage collections and time.

At this point you would definitely want to consider changing your settings and allocating more memory to see how that affects performance. Visualgc can also help you quickly find trouble spots. For example you know that when you run a certain template you see a degradation in performance and a sharp increase in memory usage in your application. If you run this same process while using visualgc you can see if the JVM is actually cleaning up the memory used in this process or not. If it is not then you have a serious issue have to do with a memory leak that could be caused by a number of things but visualgc and monitoring the JVM will give you a instance validation of this problem.