Evaluation

Evaluation

The experimentation phase requires a fair amount of tweaking. Both the values of the votes and the weights of the weapon properties need to be adjusted. Both these values could be handled together, because there is no need for weights if the votes are floating-point numbers . But keeping them separate makes it easier to achieve the correct output by incremental adjustment. Therefore, we can consider the votes as integers, worrying only about their relative importance. Then, the weights are tuned independently of the weapon properties.

The experimentation phase involves observing the weapon choices during the play. When a decision is dubious, the voting is broken down to see what went wrong. It's possible to see which weapon property had too many votes and isolate the features that contributed most. The feature is adjusted until the total votes are reasonable. Then, when the votes seem balanced, it's a matter of adjusting the weights for each weapon property. These values can be adjusted by identifying the property that contributes too strongly to the final fitness and reducing its weight as appropriate.

This methodology makes it somewhat easier to adjust all the values involved. The voting system also has the advantage of being easy to extend. New features can be added easily, and mostly independent from other existing features. The resulting system does relatively well at selecting weapons; although the behavior is no doubt suboptimal, it's practically indistinguishable from human behavior.

On the downside, setting up a voting system often requires this kind of experimentation because the results of the votes are not necessarily known. In this case, it's not possible to use supervised learning techniques. There's also a problem with the assumptions made; indeed, rarely do player skills match the exact statistics of the weapons. This implies the behaviors are not customized to individual player abilities .