8.4 Setting HIGHOFFLOAD and LOWOFFLOAD thresholds

---

8.4 Setting HIGHOFFLOAD and LOWOFFLOAD thresholds

The setting of the high and low offload thresholds have a significant impact on the performance and availability of the log stream. The values that you specify differ, depending on whether the log stream connector is a funnel-type or an active one.

8.4.1 High and low offload thresholds for funnel-type log streams

The low offload threshold for a funnel-type log stream should usually be zero. Most situation will be satisfied by having the data offloaded to a low offload value of zero.

The setting of the high offload threshold usually can be calculated and adjusted according to your environment.As a general rule, a high offload threshold of 70% should provide acceptable performance and availability. The most important thing is that it should be possible to move all the data below HIGHOFFLOAD to the offload data sets in less time than it would take the application to fill the space between HIGHOFFLOAD and the interim storage filling completely.

This is shown in Figure 8-1. In this example, the structure is 50 MB, and the HIGHOFFLOAD threshold is set to 70%, meaning that there is 15 MB above the threshold. The application writing to this structure is generating 450 KB of log data per second, meaning that it would take 33 seconds to fill up the structure. In order to provide acceptable performance, it must be possible to offload the 35 MB of data below the HIGHOFFLOAD threshold in less than 33 seconds. If it turns out that you cannot offload the data in that time, you can decide to decrease the HIGHOFFLOAD threshold, meaning that there is less data to move every time HIGHOFFLOAD threshold is reached, and also that there is more storage above the threshold, meaning that it will take longer to fill the structure.

Figure 8-1: Calculating High Threshold value

8.4.2 High and low offload thresholds for active log streams

Unlike funnel-type log streams, you specifically do not want to move all data out of interim storage. Rather, you want to keep data in interim storage for as long as possible, within the constraint of always having to have available space in interim storage for new writes.

The HIGHOFFLOAD threshold for an active log stream should not be higher than 80%. However, there is no good reason for specifying it any lower than this value.

There are two schools of thought about the value that should be used for the LOWOFFLOAD threshold. One holds that it should be set to 60%, striking a balance between keeping as much data in interim storage as possible, while at the same time ensuring that each offload frees up enough data to ensure that another offload will not be required too soon. The other holds that LOWOFFLOAD should be set as high as 75%. The reason for this is that it minimizes the amount of processing required by the offload and increases the amount of data that can be kept in interim storage. However, measurements have shown that any difference in performance between the two values falls within the normal margin of error.

---