11.5 Evaluation

---

11.5 Evaluation

All these performance measurements mean nothing unless there is some relationship associated with the measure. For example, how do we know if for some given metric it is important to maximize its value or minimize its value? To make sound judgments we must understand the measures we are taking and what their relationship is to system values, as shown in Figure 11.6. For example, for a CPU do we wish to have a high number of instructions per second or a low number? Are we looking for medians or modes? It makes a difference in how the results get interpreted. To make sound decisions about how to interpret the measurements requires that we understand how they are related to each other. For example, high disk utilization may map to low system throughput. Or high CPU utilization may map to high throughput. It is important to know which is which in order to make sound decisions.

Figure 11.6: Metrics versus usefulness.

How do we know what is adequate or good performance-especially if the system for which we are considering this question does not exist as yet. The problem is that of setting performance requirements for an as yet non-existing system. Typically we specify requirements in a nonquantitative way. We may make statements such as: The system should have low overhead, the memory and processor speeds should be synchronized, there should be a low probability of failure, and so on. In all of these cases we have stated qualitative requirements, which may be very hard to measure and realize. They are nonspecific, nonmeasurable, and, therefore, unacceptable. To change this the analyst should look at what the system will be required to do, and what capacity would be needed for a typical system with the same loads. We may also wish to add in some growth factor, say 100 percent. Therefore, we would specify a system that will meet our processing requirements and still have growth capacity equal to that being used in the present system.

---