10.9 Entropy

The specific implications for software design of the execution profile can be drawn from information theory. In this case, each scenario under which the program can execute has its own set of execution profiles. Each of the execution profiles can be measured in terms of its functional entropy, defined as follows:

where represents the execution profile of the k^th functionality.

The design entropy that represents the particular manifestation of all of the program's functionalities, as articulated in the functional design specifications, can be described as:

where m is the number of functionalities and q_i represents the functional profile.

The point of maximum entropy will occur when all modules will execute with an equal probability, in which case h_max = nlog₂n. This point of maximum entropy will occur in a circumstance where all modules of a program are executed for precisely the same amount of time under a given input scenario. It is a point of maximum surprise. There is no way of guessing where the program is likely to be executing at any point. At the other extreme, the point of minimum entropy, 0, is the point at which the program will execute in only one of its modules. The probability of executing this one module will be 1.0. The probability, therefore, of executing any other module will be 0.

The concept of entropy can also be used to evaluate a design in terms of its modularity. We can see from the standpoint of the entropy measure that the maximum effect of a decision to increase the number of modules in a design from n to n+1 is basically the difference between log₂n and log₂(n + 1). For small n, this difference might be substantial. For large n, the incremental effect of the additional module is not that great.