10.4 Estimates for Profiles

10.4 Estimates for Profiles

Now we know what our measurement objectives are. We wish to know the operational, functional, and module profiles. Unfortunately, these data are known only to Nature, who is not anxious to share her knowledge of these profiles with us. We can never know the real distribution of these three things. We can, however, develop reasonable estimates for the profiles.

The focus will now shift to the problem of understanding the nature of the distribution of the probabilities for various profiles. We have thus far come to recognize these profiles in terms of their multinomial nature. This multinomial distribution is useful for representing the outcome of an experiment involving a set of mutually exclusive events. Let

where Si is one of M mutually exclusive sets of events. Each of these events would correspond to a program executing a particular module in the total set of program modules. Further, let

Pr(Si) = wi

wT = l - wl - w2 - ··· - wM

where T = M + 1. In this case, wi is the probability that the outcome of a random experiment is an element of the set Si. If this experiment is conducted over a period of n trials, then the random variable Xi will represent the frequency of Si outcomes. In this case, the value n represents the number of transitions from one program module to the next. Note that:

XT = n - X1 - X2 - ··· - Xm

This particular distribution will be useful in the modeling of a program with a set of k modules. During a set of n program steps, each of the modules can be executed. These, of course, are mutually exclusive events. If module i is executing, then module j cannot be executing.

The multinomial distribution function with parameters n and w = (w1,w2,...,wT) is given by:

where xi represents the frequency of execution of the ith program module.

The expected values for the xt are given by:

the variances by:

Var(xi) = nwi(1 - wi)

and the covariance by:

Cov(wi, wj) = -nwiwj, i j

We would like to come to understand, for example, the multinomial distribution of a program's execution profile while it is executing a particular functionality. The problem here is that every time a program is run, we will observe that there is some variation in the profile from one execution sample to the next. It will be difficult to estimate the parameters w = (w1,w2,...,wT) for the multinomial distribution of the execution profile. Rather than estimating these parameters statically, it would be far more useful to us to get estimates of these parameters dynamically as the program is actually in operation. In essence, we would like to watch the system execute, collect data, and stop when we know that we have enough information about the profiles to satisfy our needs.

Unfortunately, for the multinomial distribution, the probabilities are the parameters of the distribution. We cannot know or measure these. We can, however, observe the frequency that each module has executed. Thus, it is in our interest to choose a probability distribution whose parameters are related to the things that we can measure.

To aid in the understanding of the nature of the true underlying multinomial distribution, let us observe that the family of Dirichlet distributions is a conjugate family for observations that have a multinomial distribution (see Reference3). The probability density function for a Dirichlet distribution, D(α,αT), with a parametric vector α =(α1,α2,...αk-1) where (α1>0; i = 1,2, ...,k-1) is:

where (wi > 0; i = 1,2,...,M) and . The expected values of the wi are given by:

(1) 

where . In this context, α0 represents the total epochs. The variance of the wi is given by:

(2) 

and the covariance by:

Obtaining confidence intervals for our estimates of the parameters for the Dirichlet distribution is not a very tractable problem. To simplify the process of setting these confidence limits, let us observe that if w = (w1,w2,...,wM) is a random vector having the M-variate Dirichlet distribution, D(α,αT), then the sum z = w1 +...+ wM has the beta distribution:

or alternately:

where γ = α1 + α2 +...+αM.

Thus, we can obtain 100(1 - α) percent confidence limits for:

μT - a μT μT + b

from

(3) 

and

(4) 

Where this computation is inconvenient, let us observe that the cumulative beta function, Fβ, can also be obtained from existing tables of the cumulative binomial distribution, Fb, by making use of the knowledge from Raiffa [4], [5] that:

(5) 

and

Fb(αT |1 - (μT + b),γ + αT) = Fβ(μT + b|γ,αT)

The value of using the Dirichlet conjugate family for modeling purposes is twofold. First, it permits us to estimate the probabilities of the module transitions directly from the observed module frequencies. Second, we are able to obtain revised estimates for these probabilities as the observation process progresses.

Let us now suppose that we wish to obtain better estimates of the parameters for our software system, the execution profile of which has a multinomial distribution with parameters n and W = (w1,w2...,wM), where n is the total number of observed module transitions and the values of the wl are unknown. Let us assume that the prior distribution of W is a Dirichlet distribution with a parametric vector α = (α1,α2,...,αM), where (αi>0; i = 1,2,...,M). In this case, αi is the observed frequency of execution of module i over epochs. If we were to let the system run for an additional, say, epochs, then we would get better estimates for the parameters for the cost of the observation of these new epochs. Then the posterior distribution of W for the additional observations X = (x1,x2,...,xM) is a Dirichlet distribution with parametric vector α* = (α1 + xl,α2 + x2,...,αM + xM) (see also Reference 5).

As an example, suppose that we now wish to measure the activity of a large software system. At each epoch (operational, functional, or module) we will increment the frequency count for the event. As the system makes sequential transitions from one event to another, the posterior distribution of W at each transition will be a Dirichlet distribution. Further, for i = 1,2....T the ith component of the augmented parametric vector a will be increased by one unit each time a new event is expressed.

[4]Raiffa, H. and Schlaifer, R., Applied Statistical Decision Theory, Studies in Managerial Economics, Harvard University, Boston, 1961.

[5]DeGroot, M.H., Optimal Statistical Decision, McGraw-Hill, New York, 1970.



Software Engineering Measurement
Software Engineering Measurement
ISBN: 0849315034
EAN: 2147483647
Year: 2003
Pages: 139

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net