What Makes HotSpot Better?

The HotSpot performance engine, which is encapsulated in both the Client- and Server VM, has dramatic improvements in just about all possible areas: from its fast thread synchronization techniques to its more reliable and faster performance garbage collection. Take a look at several new performance features of the HotSpot performance engine:

• On-the-Fly Adaptive Compilation

• Method Inlining

• Improved and Redesigned Object Layout

• Fast and Fully Accurate Garbage Collection

• Fast Thread Synchronization

On-the-Fly Adaptive Compilation

HotSpot takes advantage of the 80/20 rule that exists for more real-world applications. That is that most real-world applications spend 80% of time in 20% of the code. This simple assumption allows the adaptive compilation to work.

What happens is that when a Java application using the HotSpot technology is loaded, an interpreter starts running the bytecode. While it starts running the code, it looks for sections of the application that seem to be getting more attention than other areas. These areas are commonly known as hot spots. These hot spots are actually compiled after the interpreter detects them. After they are compiled, the compiled versions are used rather than the interpreted bytecode. This ensures that the sections of the application that are being executed over and over get the most attention from the compiler. With the JIT compiler, all code was compiled when accessed, regardless of how often it was accessed. This meant that sections of the application that were rarely used, were getting as much attention as the hot spots.

There are several benefits to this performance feature. First, programs generally start up faster because there is less compilation being performed at the start of the application run compared to JIT-type compilers. Secondly, compilation tends to be spread out over the life of the application. This has the effect of reducing the noticeable pauses with previous compilers and interpreters.

Finally, because less time is spent on compiling the entire application, more time can be spent optimizing the code. For example, more information can be obtained for optimizes like method inlining, which is talked about in the next section.

Method Inlining

Method inlining is when the compiler notices that a method call is being made that can be handled in a different way and makes changes to the execution instructions. These changes do not change the basic implementation or outcome of the method call, but reduce the overhead when making the method call.

For example, suppose you have a method that just decrements an argument passed in to the method. The compiler can detect this method and just decrement one from the argument instead if making the method call. This will increase the speed of the application, because the act of setting up the method call, making the call, and getting a return takes time and memory to execute. Remember, method inlining is trying to reduce the time it takes to execute instructions.

The problem with method inlining lies in a benefit of using Java as an object-oriented language. In Java, a base class can implement a method. Then subclasses of the base class can override that method. Because classes can be loaded dynamically, the virtual machine can never be 100% sure which class will implement the method at runtime. This is sometimes referred to as Dynamic Dispatch.

The HotSpot virtual machines uses what is known as dynamic deoptimizaton. This means that if the code has already been compiled and the virtual machine detects that something has changed with respect to that code, the virtual machine will revert to the interpreted code. If the application continues to execute that section of code, it will then be recompiled and go on to use the compiled version until something else changes with it.

Improved and Redesigned Object Layout

The HotSpot virtual machine also has removed the concept of handles, which was a level of indirection used to access objects in memory. In previous versions of the virtual machine, a table of handles was maintained. These handles, among other things, maintained a pointer to an object. If object X had a conceptual reference to object Y, what really was going on was that object X had a reference to a table entry that then pointed to object Y. The purpose of this level of indirection was to allow for faster pointer updates. Imagine if 10 objects pointed to object Y and then object Y changed, only the table entry would have to change. If instead all 10 objects pointed to object Y directly, then each of the 10 pointers to object Y would have to change. This was primarily done in earlier versions to help the garbage collector quickly find unreferenced objects and to more easily move objects around to prevent fragmentation.

However, because of the nature of the garbage collection process, each object basically had to be traversed anyway and this process turned out not to be much slower than the table and handle method. So with the HotSpot virtual machine, direct pointers are used to reference objects instead of handles. This is similar to the way the C language handles object references, and the speed is much faster.

Fast and Fully Accurate Garbage Collection

Prior to the 1.3 release of the HotSpot virtual machine, the garbage collection process was a "conservative" or partially accurate collector. This means that the virtual machine would err on the side of thinking something had a valid reference to an object. This approach is must easier to implement than a fully accurate garbage collector because it really didn't need to keep accurate records of which objects were alive and which were dead. If it couldn't figure it out, the collector would assume that an object had a reference to it, causing it not to be collected.

One of the major problems with this approach is that memory leaks can happen because some objects that have no references to it might not be collected if the garbage collector gets confused . Also, a conservative garbage collector needs to use handles to refer to objects because when it moves objects around, it needs to know about all the live and dead objects. This might not be the case with a conservative collector due to the problems that were mentioned before.

With the release of the HotSpot 2.0 virtual machine, the garbage collector is a fully accurate model. This means that it is aware at all times which objects are inaccessible and can be reclaimed. It is also aware which objects can be relocated so that memory can be compacted and fragmentation can be reduced or removed.

Other garbage collector features within the virtual machine increase functionality and performance. These features mainly deal with objects that are long-lived and are still being referenced. The virtual machine separates the short-lived objects and the long-lived objects. This helps because if an object is around for awhile, chances are that it will stay around longer. The virtual machine deals with these objects differently than ones that are constantly being created and then unreferenced.

Fast Thread Synchronization

The HotSpot virtual machine offers a "fully preemptive" thread model using the host operating system's thread model. In previous versions of the virtual machine, a single native thread might have been multiplexed to handle multiple Java threads.

The problem this caused was that if one of the Java threads is blocked for I/O or some other reason, all the other Java threads were also blocked and prevented from doing any work. In the new version of the HotSpot virtual machine, each Java thread maps to a single native OS thread. In this case, if a Java thread is blocked waiting on I/O, it can be preempted and another thread can execute.

There are also some optimizations done for threads that enter a synchronized method uncontested. The performance for this access is improved due to assumptions and optimizations done by the virtual machine.