6.3 Improving Application Performance | The OReilly Java Authors

I l @ ve RuBoard

The practices outlined in this section focus on client performance. That is, we assume there is a client application, probably being used by an end user , which talks to a server using RMI. The goal of these practices is to improve both the performance and the perceived performance of the client application.

Perceived performance is a strange thing. The goal in improving perceived performance is improving application responsiveness. That is, when the user clicks a button, the application does the right thing, and it does the right thing quickly . Most practices that improve performance also improve perceived performance. But the converse is not true ”improving application responsiveness can actually degrade overall performance, or at least cause the server to work harder. But that's often a price worth paying if it means the end user is happier .

This section isn't as RMI-specific as the previous two sections because, when you get right down to it, all enterprise applications have a fairly similar structure, and a lot of optimization practices are fairly general.

6.3.1 Cache Stubs to Remote Servers

One of the practices in Section 6.2 was entitled "Always Use a Naming Service." This is sound advice ”it does make the application more robust. When followed na vely, it also makes the application much slower. If you simply replace a remote method call (to a server) with two remote method calls (one to the naming service and then one to the server), you deserve whatever criticism comes your way.

A much better solution is to implement a centralized cache for stubs for remote servers. This cache should meet the following requirements:

It takes logical arguments, such as server names , and returns a stub. As much as possible, the rest of the code should not know about the structure of the application or the location of individual servers. Code is much more robust, and much easier to read, when knowledge about the application topology is encapsulated.
If the cache doesn't have a stub already, it fetches the stub (and does so in a way that's transparent to the rest of the application).
The cache expires stubs that haven't been used in a while. Otherwise , the cache will keep remote server stubs until the client application shuts down. Because having a live stub counts as a reference to the distributed garbage collector, holding on to stubs for long periods of time effectively prevents the server JVM from cleaning up unused resources.
It has a way for you to flush bad stubs (e.g., stubs to servers that no longer exist). When you discover that a stub is bad (e.g., attempting to make a remote call on the associated server consistently throws an instance of RemoteException ), you should remove the stub from the cache right away, instead of waiting for it to expire.

If you implement a stub cache that meets these four criteria, remote method call invocations will look like the following code snippet ( assuming that your implementation of a cache has get and remove methods ):

 try { MyServer stubToRemoteServer = (MyServer) cache.get(LOGICAL_NAME_FOR_SERVER); stubToRemoteServer.performMethod( . . . ); }  catch {RemoteException remoteException) { cache.remove(LOGICAL_NAME_FOR_SERVER); }

This has many benefits. For one, it's clear code. If the cache has the stub (e.g., if any part of the application has already talked to the server in question), the naming service is bypassed. And if the stub points to a server that is no longer running, the stub is immediately removed.

The next step in using a stub cache is to integrate it with the retry loop discussed earlier. Here's a very simple integration to illustrate the idea:

 public void wrapRemoteCallInRetryLoop( ) { int numberOfTries = 0; while (numberOfTries < MAXIMUM_NUMBER_OF_TRIES) { numberOfTries++; try {  MyServer stubToRemoteServer = (MyServer)                                                  cache.get(LOGICAL_NAME_FOR_SERVER);  doActualWork(stubToRemoteServer); break; } catch (RemoteException exceptionThrownByRMIInfrastructure) {  cache.remove(LOGICAL_NAME_FOR_SERVER);  reportRemoteException(exceptionThrownByRMIInfrastructure); try { Thread.sleep(REMOTE_CALL_RETRY_DELAY); } catch (InterruptedException ignored) {} } } }

This attempts to get the stub and then makes the remote call. If the call fails, the stub is flushed and, on the second try, a new stub is fetched from the naming service. This is good. In most cases, the cache has the stub, and the overhead of the cache is strictly local. In return for the overhead of maintaining a local cache, you've eliminated most of the calls to the naming service.

Using a stub cache inside the retry loop also lets you gracefully handle the cases when a server is restarted or migrated . In these cases, the stubs in the cache won't work, and attempting to use them will cause instances of RemoteException to be thrown. Logically, the client performs the following sequence of operations:

The client fetches the stub from the cache and attempts to make a remote method call.
The call fails because the server the stub references is no longer running.
The client removes the stub from the cache.
The retry loop kicks in, and the client tries again.
Because the cache is empty, the cache fetches a new stub from the naming service.
Because the new stub points to the new server, everything succeeds.

Combining a stub cache with a retry loop is both efficient and robust!

6.3.2 Consider Caching Return Values from Distributed Calls

At one company for which I worked, my first task was described in the following way: "This call is too slow. See if you can make it faster."

I looked at the call. It was slow. It involved 18 separate modules on the server and implicitly depended on about 40,000 lines of code, and all I could think was, "Oh yeah. I'll optimize this in my first week on the job."

Because the task was impossible , I decided to cheat. I thought, "Hmmm. If I can't make the call faster, I can at least use a cache to try and make the call less often." This turned out to be good enough for the application in question, and is often good enough in distributed applications.

The benefits of caching return values on the client side are:

Faster performance for a given sequence of operations: Instead of making a call to the server, which involves network and marshalling overhead, you retrieve the value from an in-process cache. Unless you're doing something truly stupid, this should be faster.
More reliable and predictable performance for a given sequence of operations: In addition to the performance benefit, looking in a local data structure is more reliable than calling a remote server, and it has a predictable level of overhead (unlike a remote method call, which can take more time if the network is busy).
Lower bandwidth: Bandwidth is the scarcest resource for distributed applications because it is shared by all distributed applications, and because upgrading a network is a lot harder than upgrading a single server. If you don't make a remote call, you aren't using the bandwidth necessary to make the remote call.
Lower server load: If you don't call the server, the server doesn't have to handle the call, which means you've lowered the effective load on the server.

On the other hand, caching does have some major drawbacks. The two most significant are:

Potentially inaccurate data: Whenever you cache data, you run the risk of the cached values being incorrect. Sometimes this isn't very important. Sometimes it is. Generally speaking, the more important a piece of information is, or the more time-sensitive it is, the less likely you are to cache it.
The cost of maintaining the cache: Within a given sequence of operations (for example, responding when a user clicks a button), a cache improves performance. But caches have to be maintained : data has to be checked for consistency and occasionally thrown out.

Cache maintenance is usually done in a background thread so that a main thread doesn't take the hit of maintaining the cache (otherwise, perceived performance would suffer, and the "more predictable" item listed as an advantage would be false ). But nonetheless, if you're not accessing data that's been cached very often, the client-side cache might very well be a net loss in performance.

This last item is somewhat paradoxical in that, even if the cache winds up costing you more processing time than it saves, it still might be viewed as a performance improvement. This is because the cost occurs in a background thread, but the performance improvements occur in a main thread (where performance improvements are likely to be noticed).

Caching is a subtle and tricky subject. One reference worth looking at is the series of data expiration articles I wrote for onjava.com. In addition, if you're interested in the subject, you might want to keep an eye on JSR-107, the "Java Temporary Caching API," at http://jcp.org.jsr/detail/107.jsp.

6.3.3 Use Batch Methods to Group Related Method Calls

I once had lunch with a group of programmers who were visiting my company from Germany. They were from a company that was partnering with the company I was working for. We were supposed to use CORBA to handle the method calls from their system to ours. But there was a problem.

"CORBA," they confided to me, "doesn't work. It's much too expensive. It doesn't scale very well at all." I was a little surprised by this. I'd been using CORBA for a couple of years at that point, and I'd never run into scalability problems (at least, not at the scale we were talking about).

It turned out that their idea of "building a distributed application" was to "build a single-process application using standard object-oriented techniques and then stick the network in." Their testing had revealed that if an instance of Person is on one machine, and the user interface displaying information about the person is on another machine, a sequence of fine-grained calls such as getFirstName( ) , getLastName( ) , and getSSN ( ) to get the values to display on the screen has performance problems.

Once I figured out what they were complaining about, I had a solution: "Don't do that." The object decompositions that make sense in a single-process application often don't make as much sense in a distributed application. Instead, you need to carefully look at the intent behind sequences of calls and see if you can encapsulate a sequence of calls in one bigger method call (which I refer to as a batch method).

In the previous example, the sequence getFirstName( ) , getLastName ( ), getSSN ( ), . . . should really have been a call to a new method called getDisplayData( ) . Of course, getDisplayData( ) shouldn't exist for a single-process implementation of the Person class (it completely breaks encapsulation). But it has to be there for the distributed application to perform well, and that's the point of this practice.

How do you spot a potential batch method? There's no cut-and- dried way to do so (that I know of). But here are four rules of thumb for when batch methods are appropriate:

The methods you want to include in a batch method must be called in a sequence fairly often. Batch methods are an optimization and they shouldn't be created for uncommon cases.
If the methods are all for data access (with no side effects), a batch method is more likely to be appropriate.
The methods should all block the same task (and not distinct tasks). If all the method calls are associated with the same set of client-side conceptual tasks, and none of the tasks can continue until the client gets all the return values, a batch method is almost certainly appropriate.
The methods have a predetermined outcome. That is, the client will make a sequence of calls. If the client doesn't know all the arguments (for all the calls) at the time the first call is made, or if the return values for all the calls can't be computed at the time when the first call is made, the methods can't be grouped into a batch method.

6.3.4 Use a Server-Side Proxy to Manage Transactions Across Servers

Sometimes a client will make a series of calls to a server, and these calls, while conceptually a transaction, aren't easily batched, or don't feel like a batch method. A classic example of this is transferring money between two bank accounts.

If you have two distinct servers (one for each account), this involves the following four steps:

Opening a transaction (presumably using a transaction management server)
Withdrawing money from one server
Depositing the money in another server
Committing the transaction

This sequence of calls should not be executed from a client application for two reasons. The first is that client applications are often deployed across a WAN, which is slow ( especially compared to a datacenter managed by IT personnel). Making four concurrent remote method calls under those conditions could lead to significant performance problems.

The second reason is that logic about transactions will most likely change. Even something as simple as "We've decided to log all money transfers from now on" would force you to redeploy a new version of the client that would call a logging method on a server somewhere, which is very bizarre in and of itself.

The solution is to use a server-side proxy for the client. A server-side proxy is a server, running "near" the other servers, which is client-specific and whose sole role is to manage complex transactions for the client. If you're using a server-side proxy, the previous calling sequence turns into a single call to the proxy's transferMoney method. The proxy still has to perform the four remote method calls, but it does so on a LAN inside a well-managed environment. Figure 6-1 illustrates the trade-off.

Figure 6-1. Using a server-side proxy

6.3.5 Use Asynchronous Messaging Wherever Possible

Most remote method calls are synchronous: the calling thread stops and waits for a response from the server. An asynchronous method call is one in which the caller doesn't wait for a response, or even wait to know if the call succeeded. Instead, the calling thread continues processing.

RMI doesn't directly support asynchronous calls. Instead, you have to use a background thread to make the call. In Example 6-2, the calling thread creates a command object and drops it off in a background queue for execution. This leaves the calling thread free to immediately resume processing, instead of waiting for a response, as in the following code snippet:

 BackgroundCallQueue_backgroundQueue = new BackgroundCallQueueImpl(  ); //  . . .  _backgroundQueue.addCall(new SpecificCallToServer( . . . ));

Example 6-2 is a sample implementation of BackgroundCallQueue and BackgroundCallQueueImpl .

Example 6-2. BackgroundCallQueue.java and BackgroundCallQueueImpl.java

 public interface BackgroundCallQueue {     public void addCall(RemoteMethodCall callToAdd); } public class BackgroundCallQueueImpl implements BackgroundCallQueue {     private LinkedList _pendingCalls;     private thread _dispatchThread;     public BackgroundCallQueueImpl (  ) {         _stopAcceptingRequests = false;         _pendingCalls = new LinkedList(  );         _dispatchThread = new Thread(this, "Background Call Queue Dispatch Thread");         _dispatchThread;.start(  );     }     public synchronized void addCall(RemoteMethodCall callToAdd) {         _pendingCalls.addCall(  );         notify(  );     }     public void run(  ) {         while (true) {             RemoteMethodCall call = waitForCall(  );             if (null!=call ) {                 executeCall(call);             }         }     }     private synchronized RemoteMethodCall waitForCall(  ) {         while (0== _pendingCalls.size(  )) {             wait(  );         }         return (RemoteMethodCall) _pendingCalls.removeFirst(  );     }     private void executeCall(RemoteMethodCall call) {         //  . . .      } }

This isn't very complicated code, but it does offer two very significant advantages over synchronous method calls. The first is that it decreases the time a main thread spends sending remote messages. Imagine, for example, that a user clicks on a button, and, as a result of that click, the server needs to be told something. If you use a synchronous method call, the button processing time (and, hence, the perceived performance of the application) will include the time spent sending the remote method call (and the time the server spends processing the call). If you can make the call asynchronously, in a background thread, the application isn't any faster or more efficient, but the user thinks it is.

The second reason is that, once you've moved to a model where requests are dropped off into a queue, you can tweak the queue and make performance improvements without altering most of the client code. For example, if you are making a lot of calls to a single method on a server, you can group these calls and make them in a single call. For example, instead of 100 calls to:

 server.patientGivenMedication(Patient patient, Medication medication, long time,  HealthCareProvider medicationSource);

you might have 1 call to:

 server.patientsGivenMedication(ArrayList medicationEvents);

Because each remote method call contains information about all the relevant classes involved, this will dramatically reduce both marshalling time and bandwidth. Instead of marshalling and sending information about the Patient class 100 times, you will send it only once.

There are two major downsides to putting messages in a background queue. The first is that your code will be harder to debug. Decoupling the source of the remote method call from the time the remote call is made makes it harder to trace the source of logical errors. For example, if a command object has the wrong value for an argument, it's harder to track down the source of the error.

The second problem with putting messages in a background queue is that it's harder to report failures (and harder to respond to them). If a user clicks a button and therefore thinks of an operation as "done," it can be disconcerting for him to find out later on that the operation failed.

Given all this, the question is: when should you use asynchronous messaging? Here are three indicators that a method can be put safely into a background queue:

If the method returns void and throws no exceptions (other than RemoteException ). Methods with this sort of signature tend to be information-sending methods rather than information-retrieval methods or methods that are requesting an action. And, hence, they can be safely postponed and performed in the background.
If there is an associated method to find out the status of an operation at a later time. For example, consider a method to submit a print request to a print server. It might have a boolean return value, and it might throw all sorts of interesting exceptions. But having clicked on the Print button, the user really wants to be able to continue with his work, not wait until the method call transfers the entire file to the print server, and he wants a status view so that he can see where his document is in the print queue.
If the server can call the client back with a return value (or an exceptional condition). Certain requests aren't particularly urgent and can be performed in the background. But the user would like to know if he succeeded. Consider email, for example. When he clicks on the Send button, the user is really saying, "This can go off into the email cloud now." Whether it is sent now or three minutes from now is not really an issue. But he is assuming that if it can't be sent, he will be told.

Printing documents is also a case when callbacks can improve the user interface. A callback to let a user know that his document has been printed is a nice feature.

Using callbacks to let the user know the outcome of an event could incur some additional overhead. It replaces one bidirectional socket connection (the method call and then the return value) with two unidirectional connections (the method call is sent and then, in a later connection, the response is sent). Breaking apart messages like this is a useful technique for optimizing perceived performance, but it almost always incurs some extra overhead.

I l @ ve RuBoard