Performance Measurements

---

The carrying of safety stocks is to guard against the uncertainties and variations in demand and the forecasting of demand. These safety stocks are directly related or made directly proportional, to the standard errors of forecasts. If, as is usually the case, a stock controller is dealing with many, many items, it is the performance across the group of items that matters. Hence, the sum of the standard errors of the forecasts, measured by the sum of the root mean squared errors over all the items, is used to compare the results between the different weighting methods. The ultimate aim is to find the one best overall method for the weighting process to use for all the forms required by the bank. This decision is being made by comparing alternatives for a sample of the forms used by the bank. The process chosen on the basis of this sample is then to be applied to the full set of 300 forms. When a single time series is being considered, then the usual approach is to make the choice on the single criterion of choosing the process minimizing the root mean squared error. Over the sample of 10 items, the process giving the lowest sum of root mean squared errors may have one or two very bad performances, e.g., high percentage errors. There has to be concern that such high errors could occur for the highest demand items or for highest varying demand items in the remaining 290 forms not considered in the sample. It may be better to choose another method that is slightly worse overall for the sample but which has a lower worst case performance over the sample. For this reason, a number of other indicators were taken into account in the selection process, as given below. The weighting process is taken over the three different forecasting methods, one for each forecasting group, selected for that item.

OP	Number of items for which the weighting method outperforms or equals the performance of the actual best of the individual three forecasting methods.
BP	Number of items for which the weighting method gives the best performance of the four weighting alternatives.
WP	Number of items for which the method performs the worst.
MPI	Maximum individual percentage improvement over all items in the root mean squared error (RMSE) for the weighting method over the RMSE for the best actual individual forecasting method over the test period.
WPP	Maximum percentage that the root mean squared error of the method is worse than the best individual forecast for any item.
SRMSE	Sum of root mean squared errors.

---