Evaluating Performance | Implementing Backup and Recovery: The Readiness Guide for the Enterprise

< Day Day Up >

Measuring NetBackup Performance

Once you have established an accurate metric as described here, you can measure the current performance of NetBackup and your system components to compile a baseline performance benchmark. Once you have a baseline, you can apply changes in a controlled way. By measuring performance after each change, you can accurately measure the effect of each change on NetBackup's performance.

The following topics are covered in this section:

Controlling system variables for consistent testing conditions
Evaluating performance through the Activity Monitor and the All Log Entries report
Evaluating system components, such as your CPU and memory

Controlling System Variables for Consistent Testing Conditions

For reliable performance evaluation, eliminate as many unpredictable variables as possible in order to create a consistent backup environment. Only a consistent environment will produce reliable and reproducible performance measurements. Some of the variables to consider are described below as they relate to the NetBackup server, the network, the NetBackup client, or the data itself.

Server Variables

Of course, it is important to eliminate all other NetBackup activity from your environment when you are measuring the performance of a particular NetBackup operation. One area which you must consider during this process is the automatic scheduling of backup jobs by the NetBackup scheduler.

When policies are created, they are usually set up to allow the NetBackup scheduler to initiate the backups. The NetBackup scheduler will initiate backups based on the traditional NetBackup frequency-based scheduling or on certain days of the week, month, or other time interval. This process is called calendar-based scheduling and is new in NetBackup 4.5. As part of the backup policy definition, the Start Window is used to indicate when the NetBackup scheduler can start backups using either frequency-based or calendar-based scheduling. When you perform backups for the purpose of performance testing, this setup might interfere since the NetBackup scheduler may initiate backups unexpectedly; especially if the operations you intend to measure run for an extended period of time.

The simplest way to prevent the NetBackup scheduler from running backup jobs during your performance testing (assuming you are testing the performance of a backup job), is to create a new policy specifically for use in performance testing and to leave the Start Window field blank in the schedule definition for that policy. This prevents the NetBackup scheduler from initiating any backups automatically for that policy. After creating the policy, you can run the backup on demand by using the Manual Backup command from the NetBackup Administration Console.

To prevent the NetBackup scheduler from running backup jobs unrelated to the performance test, you may want to set all other backup policies to inactive by using the Deactivate command from the NetBackup Administration Console. Of course, you must reactivate the policies to start running backups again.

You can use a user-directed backup to run the performance test as well. However, we suggest using the manual backup option for a policy, (or immediate backup, as it is sometimes called) instead of a user-directed backup to more closely simulate initiation of a backup job by the NetBackup Scheduler. Additionally, if you use a manual backup, the backup policy will contain the entire definition of the backup job, including the clients and files that are part of the performance test. Confining the definition of the backup job to a single location makes it easier to manage changes to the definition, which in turn makes reliable performance test results more likely.

You may want to consider changing the wakeup interval for the NetBackup Scheduler to a larger value for the duration of performance testing. Use the Global NetBackup Attributes tab of the Master Server Properties dialog to modify the wakeup interval. Access the Master Server Properties dialog through the Host Properties node of the NetBackup Administration Console in release 4.5, or the Configure NetBackup GUI in earlier releases. The wakeup interval controls how often the NetBackup Scheduler examines the policy definitions on the master server to determine if there are any new backup jobs to start. If there are a large number of NetBackup policies defined for the master server, this action by the Scheduler may affect the performance test. Setting the wakeup interval to a large value, such as 1440 (24 hours) will avoid any impact of this processing on the performance test.

Be aware that changing the wakeup interval will not reflect performance in your actual production environment. For more realistic results, use the wakeup interval specified in your production environment.

Before you start the performance test, check the Activity Monitor to make sure there is no NetBackup processing currently in progress. Similarly, check the Activity Monitor after the performance test for unexpected activity (such as an unanticipated restore job) that may have occurred during the test.

Additionally, check for non-NetBackup activity on the server during the performance test and try to reduce or eliminate it.

Network Variables

Network performance is key to achieving optimum performance with NetBackup. Ideally, you would use a completely separate network for performance testing to avoid the possibility of skewing the results by encountering unrelated network activity during the course of the test.

In many cases, a separate network is not available. Ensure that non-NetBackup activity is kept to an absolute minimum during the time you are evaluating performance. If possible, schedule testing for times when backups are not active. Even occasional short bursts of network activity may be enough to skew the results during portions of the performance test. If you are sharing the same network as any production backups occurring for other systems, you must account for this activity during the performance test.

Another network variable you must consider is host name resolution. NetBackup depends heavily upon a timely resolution of host names to operate correctly. If you have any delays in host name resolution, including reverse name lookup to identify a server name from an incoming connection from a certain IP address, you may want to eliminate that delay by using the HOSTS file for host name resolution on systems involved in your performance test environment.

Client Variables

Make sure the client system is in a relatively quiescent state during performance testing. A lot of activity, especially disk-intensive activity such as virus scanning, will limit the data transfer rate and skew the results of your tests.

One possible mistake is to allow another NetBackup server, such as a production backup server, to have access to the client during the course of the test. This may result in NetBackup attempting to backup the same client to two different servers at the same time, which would severely impact the results of a performance test in progress at that time.

Different file systems have different performance characteristics. For example, comparing data throughput results from operations on a FAT file system to those from operations on an NTFS system may not be valid, even if the systems are otherwise identical. If you do need to make such a comparison, factor the difference between the file systems into your performance evaluation testing, and into any conclusions you may draw from that testing.

Take care when OTM (Open Transaction Manager) is enabled to back up open files on the client during performance testing. OTM will delay for a certain amount of time waiting for the disk drive to quiesce. This delay can vary from one performance test run to the next, causing unreliable results. You may want to disable OTM on the client during performance testing if it is not needed to back up open files and avoid this unpredictable delay.

Data Variables

Monitoring the data you are backing up improves the repeatability of performance testing. If possible, move the data you will use for testing backups to its own drive or logical partition (not a mirrored drive), and defragment the drive before you begin performance testing. For testing restores, start with an empty disk drive or a recently defragmented disk drive with ample empty space. This will help reduce the impact of disk fragmentation, etc., on the NetBackup performance test run and yield more consistent results between test runs.

Similarly, for testing backups to tape, always start each test run with an empty piece of media. You can do this by expiring existing images for that piece of media through the Catalog node of the NetBackup Administration Console (new in release 4.5), or by running the bpexpdate command. Another approach is to use the bpmedia command to freeze any media containing existing backup images so that NetBackup selects a new piece of media for the backup operation. This step will help reduce the impact of tape positioning, etc., on the NetBackup performance test run and will yield more consistent results between test runs. When you test restores from tape, always restore from the same backup image on the tape to achieve consistent results between test runs.

In general, using a large data set will generate a more reliable and reproducible performance test than a small data set. A performance test using a small data set would probably be skewed by startup and shutdown overhead within the NetBackup operation. These variables are difficult to keep consistent between test runs and are therefore likely to produce inconsistent test results. Using a large data set will minimize the effect of start up and shutdown times.

Design the makeup of the dataset to represent the makeup of the data in the intended production environment. For example, if the data set in the production environment contains many small files on file servers, then the data set for the performance testing should also contain many small files. A representative test data set will more accurately predict the NetBackup performance that you can reasonably expect in a production environment.

The type of data can help reveal bottlenecks in the system. Files consisting of non-compressible (random) data cause the tape drive to run at its lower rated speed. As long as the other components of the data transfer path are keeping up, you may identify the tape drive as the bottleneck. On the other hand, files consisting of highly-compressible data can be processed at higher rates by the tape drive when hardware compression is enabled. This scenario may result in a higher overall throughput and possibly expose the network as the bottleneck.

Many values in NetBackup provide data amounts in KB and rates in KB/Sec. For greater accuracy, divide by 1024 rather than rounding off to 1000 when you convert from KB to MB or from KB/Sec. to MB/Sec.

< Day Day Up >