5.6 Some example probability distributions

---

5.6 Some example probability distributions

In this section, we will examine some discrete and some continuous probability distributions that will help to solidify the basic probability theory of the previous sections. Many of these distributions are commonly used to model real-world processes and to help arrive at estimates for quantities of interest in real-world systems. We will discuss the properties of each distribution and we will also discuss the process of deriving random deviates, given a certain distribution that models a real-world process.

5.6.1 Uniform distribution

The simplest of all probability distributions is the discrete uniform distribution. Such a distribution states that all values of the random variable are equally probable and depend only upon the number of possible outcomes of the experiment. The density function for the uniform distribution is given as:

(5.114) 

where k is the number of possible outcomes. The experiment where the random variable X = P(n), n = 1 to 6, and the event is the toss of a die that results in a discrete uniform probability distribution. A plot of the density function for the uniform distribution is shown in Figure 5.4.

Figure 5.4: Uniform density function.

The mean of the uniform distribution is found by equation (5.66) and is given by:

(5.115) 

The standard deviation is found by:

(5.116) 

5.6.2 Binominal distribution

The concept of a Bernoulli trial is important in many discrete distributions. A Bernoulli trial is an experiment in which the outcome can be only success or failure. Random variables defined on successive Bernoulli trials make up several of the discrete density functions we will discuss.

An important discrete probability distribution is the binomial distribution. This distribution results from experiments in which there are only two possible outcomes of an experiment, such as a coin toss. For the distribution, one outcome is chosen to represent success and the other failure. A binomial experiment also requires that the probability of success remains constant for successive trials, that the trials are independent, and that each experimental outcome results in success or failure. Since the trials are independent, the total probability for an experiment with x successes and n trials can be found by simply multiplying the probability of each event (see equation [5.26]). If the probability of success is given as p and if q = 1 - p, we have:

(5.117) 

For the binomial distribution, we want to find the number of successes in n independent trials, given that we know the probability of success for any individual trial. The number of successes in n trials is a combination, as given in equation (5.10). The probability of x successes in n trials, then, is the expression for the binomial distribution:

(5.118) 

Suppose we have a binomial experiment in which the outcomes of n experiments can be used to represent the random variable X, which denotes the number of successes in n trials. Thus, by equation (5.84) and by the definition of binomial random variables:

(5.119) 

Since the variance of any of the individual experiments is pq, by equation (5.100) the variance of a binomial density can be found to be:

(5.120) 

Suppose that we wish to know how many Bernoulli trials occur before the first success in a sequence of trials occurs. If the first trial yields a success and the probability of success for any trial is p, the probability of the random variable X is p. If the probability of failure is given as q = 1 - p and we have success on the second trial, we obtain a probability of pq. Extending to k - 1 failures before an eventual success, we obtain what is known as the "geometric distribution," where:

(5.121) 

Finding the expected value of the geometric density function is a bit tricky but can be accomplished as follows. By equation (5.67),

(5.122) 

The fifth line of the derivation above results because the value of q is less than or equal to 1; thus, the summation converges to 1 / (1 - q). The variance of the geometric density is not derived here but is given as:

(5.123) 

5.6.3 Poisson distribution

A widely used discrete density function that is useful for deriving statistics about the number of successes during a given time period is the Poisson distribution. The Poisson density function is popular mainly because it describes many real-word processes very well. In computer systems, requests for jobs at a CPU are often represented by a Poisson process. The Poisson density function is defined as:

(5.124) 

where the parameter μ is defined as the average number of successes during the interval. Several conditions must prevail for a Poisson density function to exist. These are that the successes for one interval are independent of the successes in any other interval, that the probability of a success during an interval of extremely short length is near zero, and that the probability of only one success during a short interval depends only upon the length of the interval. Interestingly, the mean of the Poisson distribution is part of its definition. The expected value can be found as:

(5.125) 

Now, if we let y = x - 1, we arrive at a summation of the density function from 1 to infinity, which, by equation (5.51), is equal to 1:

(5.126) 

The variance of the Poisson distribution can be found by first finding E[X(X - 1)] and then using the result in equation (5.94):

(5.127) 

The first two terms of this summation are zero, so we have:

(5.128) 

By equation (5.51), and, if we let x = y + 2, we get:

(5.129) 

By equation (5.94), we get:

(5.130) 

By equation (5.84), we get:

(5.131) 

By equation (5.127), we get:

(5.132) 

The previous density functions provide some examples of the more common discrete random variables. Distributions such as these are useful for modeling real-world processes in which the quantities of interest are countable items.

In addition to the basic discrete density functions described earlier, there are several continuous densities. Continuous density functions are characterized by the fact that the value of f(x) at any point x is zero. However, the probability that any value x lies between x and some small delta is approximately f(x) times the delta value.

5.6.4 Gaussian distribution

One of the most important continuous probability distributions, and probably the most widely used, is the "normal," or Gaussian distribution.

The density function of a normal random variable X is given as:

(5.133) 

Figure 5.5 shows a few normal curves (also known as bell curves because of their bell-like shapes). The flatter curve has a larger standard deviation than the thinner curves. The expected value of the normal curve is found as follows. By equation (5.67):

(5.134) 

Figure 5.5: A few normal curves.

If we substitute the following:

(5.135) 

we get:

(5.136) 

If, in the second integral, we replace y by the following:

(5.137) 

we clearly see the integral of a density function, which is equal to 1:

(5.138) 

The expression in the remaining integral is an odd function because of the presence of y. Since an odd function integrated over symmetric limits is zero, the mean becomes:

(5.139) 

We can find the variance of the normal distribution as follows:

(5.140) 

Making the same substitution as before, we get:

(5.141) 

Now, we can integrate by parts:

Let:

and

(5.142) 

then:

As before, the first interval equals zero, and the second integral can be shown to be:

(5.143) 

so we have:

(5.144) 

For the normal curve, the mean occurs at the mode, which is defined as the value that appears most in the distribution.

Finding the probability that a normally distributed random variable falls between two values requires the solution of the integral:

(5.145) 

This integral is not easily solvable and is best approached using numerical means. In order to be useful, however, we would need to generate a table for each value of mean and standard deviation. We would like to avoid this by having only one standard normal curve. If we make the substitution:

(5.146) 

in the previous equation, we obtain:

(5.147) 

where:

(5.148) 

The expression is equivalent to a normal distribution of mean equal to zero and standard deviation of one. Thus, we can transform any normal distribution into the standard normal curve with zero mean and a standard deviation of one. For example, let's compute the probability that any normally distributed random variable falls within one standard deviation of the mean. To do so, we need to generate some sort of table for the standard normal distribution. Table 5.1 gives values for the standard normal distribution integrated from minus infinity to x.

Table 5.1: Standard Normal Curve Values

x+0.00	x+0.01	x+0.02	x+0.03	x+0.04	x+0.05	x+0.06	x+0.07	x+0.08	x+0.09
0.50000	0.50401	0.50800	0.51199	0.51597	0.51996	0.52394	0.52794	0.53194	0.53594
0.53993	0.54391	0.54790	0.55187	0.55585	0.55981	0.56378	0.56773	0.57168	0.57562
0.57955	0.58348	0.58740	0.59130	0.59520	0.59909	0.60297	0.60684	0.61070	0.61455
0.61839	0.62221	0.62603	0.62983	0.63362	0.63739	0.64116	0.64490	0.64864	0.65236
0.65607	0.65976	0.66343	0.66709	0.67074	0.67436	0.67798	0.68157	0.68515	0.68871
0.69225	0.69578	0.69929	0.70277	0.70624	0.70970	0.71313	0.71654	0.71993	0.72331
0.72666	0.72999	0.73331	0.73660	0.73987	0.74312	0.74635	0.74956	0.75274	0.75591
0.75905	0.76217	0.76527	0.76834	0.77139	0.77442	0.77743	0.78041	0.78337	0.78631
0.78923	0.79212	0.79498	0.79783	0.80065	0.80344	0.80622	0.80897	0.81169	0.81439
0.81707	0.81972	0.82234	0.82495	0.82753	0.83008	0.83261	0.83512	0.83760	0.84006
0.84248	0.84487	0.84724	0.84959	0.85192	0.85421	0.85649	0.85874	0.86097	0.86317
0.86535	0.86751	0.86964	0.87174	0.87383	0.87589	0.87793	0.87994	0.88193	0.88389
0.88584	0.88775	0.88965	0.89152	0.89337	0.89520	0.89701	0.89879	0.90055	0.90228
0.90400	0.90569	0.90736	0.90901	0.91063	0.91224	0.91382	0.91538	0.91692	0.91844
0.91994	0.92142	0.92287	0.92431	0.92572	0.92712	0.92849	0.92985	0.93118	0.93250
0.93379	0.93507	0.93633	0.93756	0.93878	0.93998	0.94117	0.94233	0.94348	0.94460
0.94571	0.94681	0.94788	0.94894	0.94998	0.95100	0.95201	0.95300	0.95397	0.95493
0.95587	0.95679	0.95770	0.95860	0.95947	0.96034	0.96119	0.96202	0.96284	0.96364
0.96443	0.96521	0.96597	0.96672	0.96745	0.96817	0.96888	0.96957	0.97026	0.97093
0.97158	0.97223	0.97286	0.97348	0.97409	0.97468	0.97527	0.97584	0.97640	0.97695
0.97749	0.97803	0.97855	0.97906	0.97956	0.98005	0.98053	0.98100	0.98146	0.98191
0.98235	0.98279	0.98321	0.98363	0.98403	0.98443	0.98482	0.98520	0.98557	0.98593
0.98629	0.98664	0.98698	0.98731	0.98764	0.98795	0.98827	0.98857	0.98887	0.98916
0.98944	0.98972	0.98999	0.99025	0.99051	0.99077	0.99101	0.99125	0.99149	0.99172
0.99194	0.99216	0.99237	0.99258	0.99279	0.99298	0.99318	0.99337	0.99355	0.99373
0.99391	0.99408	0.99424	0.99441	0.99456	0.99472	0.99487	0.99502	0.99516	0.99530
0.99543	0.99556	0.99569	0.99582	0.99594	0.99606	0.99618	0.99629	0.99640	0.99650
0.99661	0.99671	0.99681	0.99690	0.99700	0.99709	0.99717	0.99726	0.99734	0.99742
0.99750	0.99758	0.99765	0.99773	0.99780	0.99787	0.99793	0.99800	0.99806	0.99812
0.99818	0.99824	0.99829	0.99835	0.99840	0.99845	0.99850	0.99855	0.99859	0.99864
0.99868	0.99873	0.99877	0.99881	0.99885	0.99888	0.99892	0.99896	0.99899	0.99902
0.99906	0.99909	0.99912	0.99915	0.99918	0.99920	0.99923	0.99926	0.99928	0.99930
0.99933	0.99935	0.99937	0.99939	0.99942	0.99944	0.99945	0.99947	0.99949	0.99951
0.99953	0.99954	0.99956	0.99957	0.99959	0.99960	0.99962	0.99963	0.99964	0.99966
0.99967	0.99968	0.99969	0.99970	0.99971	0.99972	0.99973	0.99974	0.99975	0.99976
0.99977	0.99978	0.99978	0.99979	0.99980	0.99981	0.99981	0.99982	0.99983	0.99983
0.99984	0.99984	0.99985	0.99986	0.99986	0.99987	0.99987	0.99988	0.99988	0.99988
0.99989	0.99989	0.99990	0.99990	0.99990	0.99991	0.99991	0.99991	0.99992	0.99992
0.99992	0.99992	0.99993	0.99993	0.99993	0.99993	0.99994	0.99994	0.99994	0.99994
0.99994	0.99995	0.99995	0.99995	0.99995	0.99995	0.99995	0.99996	0.99996	0.99996
0.99996	0.99996	0.99996	0.99996	0.99997	0.99997	0.99997	0.99997	0.99997	0.99997

Let:

(5.149) 

then:

so:

The left-hand side of equation (5.143) can be rewritten as:

(5.150) 

The values from the table for z = - 1 and 1, respectively, are:

(5.151) 

Therefore, we have:

(5.152) 

Figure 5.6 shows the selected area under the normal curve.

Figure 5.6: Selected area under a normal distribution curve.

5.6.5 Exponential distribution

A simpler continuous distribution, the exponential, is important in queuing theory and therefore is discussed here. Its main attraction is that it has the Markovian property, which states that the probability of occurrence of an event is completely independent of the history of the experiment. This characteristic is also called the "memoryless" property. The expression for an exponential distribution is given as:

(5.153) 

The graph of an exponential curve is shown in Figure 5.7.

Figure 5.7: Exponential probability density function.

We will see later that the exponential distribution, because of its Markovian property, will be useful for representing service time distributions in queuing systems.

Suppose that the time a computer user spends at a system terminal is exponentially distributed over time. The probability that the user will be at a terminal for n minutes is given as:

(5.154) 

The probability distribution function for the exponential function is shown in Figure 5.8 and given as:

(5.155) 

Figure 5.8: Exponential probability distribution function.

As with any distribution function, we can find the same result by picking the point n, representing the probability that the user will be at a terminal for less than n minutes, and using equation (5.19) to find the probability of the complementary event (see also Figure 5.8):

(5.156) 

The mean of an exponential random variable is found as:

(5.157) 

If we let:

(5.158) 

and integrate by parts where:

(5.159) 

we obtain:

(5.160) 

because:

(5.161) 

We may find the variance of the exponential random variable as follows:

(5.162) 

We can see that by equations 5.157 through 5.161, the second interval evaluates to 1/λ. The third interval evaluates to 1/λ (as shown in 5.161). To solve this first interval, we introduce the gamma function, denoted as:

(5.163) 

The gamma function can be solved for a positive value of the parameter to yield:

(5.164) 

We can now use equation (5.164) to help find the solution to equation (5.162) and arrive at the variance for an exponential distribution:

(5.165) 

5.6.6 Erlang distribution

The exponential density function is often used to represent the service time of a server at the end of a waiting line. In some cases, it is desirable to represent several identical servers with a single density function whose statistics are the same as for a single equivalent exponential server. The distribution that satisfies these conditions is called the "Erlang distribution" and is given as:

(5.166) 

with parameters λ and k. Figure 5.9 shows a graph of the Erlang density function for various values of k for a given value of λ. The mean and standard deviations of the Erlang density function are given as:

(5.167) 

Figure 5.9: Erlang density functions for selected values of k.

(5.168) 

The probability distribution function is given as:

(5.169) 

It is important to note that the expected value for the Erlang density is the same as for an exponential with the same parameter, λ, and is independent of the number of Erlangian servers (e.g., parallel servers).

---