Measuring NFS Latency

To find out how long your application program is idle and waiting for the network and the NAS server, you can run a simulation^[19] or a batch program preceded by the time command. For example, let's say you have a batch program called mytest in the /usr/local/bin directory that will perform read and write operations using data stored on the NAS server.^[20] To test its performance, start it from a shell prompt with the command:

 #time /usr/local/bin/mytest 

When the mytest program finishes executing you will receive a report such as the following:

 real   0m7.282s user   0m1.080s sys    0m0.060s 

This report indicates the amount of wall clock time (real) the mytest program took to complete, followed by the amount of CPU user and CPU system time it used. Add the CPU user and CPU system time together, and then subtract them from the first number, the real time, to arrive at a rough estimate for the latency^[21] of the network and the NAS server.

Note 

This method will only work for programs (batch jobs) that perform the same or similar filesystem operations throughout the length of their run (for example, a database update that requires updating the same field in all of the database records that is performed inside of a loop). The method I've just described is useful for arriving at the average I/O requirements of the entire job, not the peak I/O requirements of the job at a given point in time.

You can use this measurement as one of the criteria for evaluating NAS servers, and also use it to study the feasibility of using a cluster to replace your monolithic Unix server. Keep in mind, however, that in a cluster, the speed and availability of the CPU may be much greater than on the monolithic server.^[22] Thus, even if the latency of the NAS server is greater than locally attached storage, the cluster may still outperform the monolithic box.

^[19]The Expect programming language is useful for simulating keyboard input for an application.

^[20]You can use a Linux box as an NFS server for testing purposes. Alternatively, if your existing data resides on a monolithic Unix server, you can simply use your monolithic server as an NFS server for testing purposes. See the discussion of the async option later in this chapter.

^[21]The time penalty for doing filesystem operations over the network to a shared storage device.

^[22]Because processor speeds increase each year, I'm making a big assumption here that your monolithic server is a capital expenditure that your organization keeps in production for several years—by the end of its life cycle, a single processor on the monolithic server is likely to be much slower than the processor on a new and inexpensive cluster node.