Mathematics

COVARIANCE, VARIANCE OF SUMS, AND CORRELATIONS OF RANDOM VARIABLES

  The statistical parameters of the random variables of different nature using the definition of expectation of random variable is easy to obtain and understand, in the following we will find some parameters with the help of mathematical expectation of random variable.

Moments of the number of events that occur

    So far we know that expectation of different powers of random variable is the moments of random variables and how to find the expectation of random variable from the events if number of event occurred already, now we are interested in the expectation if pair of number of events already occurred, now if X represents the number of event occurred then for the events A₁, A₂, ….,A_n define the indicator variable I_i as

the expectation of X in discrete sense will be

because the random variable X is

now to find expectation if number of pair of event occurred already we have to use combination as

this gives expectation as

from this we get the expectation of x square and the value of variance also by

By using this discussion we focus different kinds of random variable to find such moments.

Moments of binomial random variables

   If p is the probability of success from n independent trials then lets denote A_i for the trial i as success so

and hence the variance of binomial random variable will be

because

if we generalize for k events

this expectation we can obtain successively for the value of k greater than 3  let us find for 3

using this iteration we can get

Moments of hypergeometric random variables

  The moments of this random variable we will understand with the help of an example suppose n pens are randomly selected from a box containing N pens of which m are blue, Let A_i denote the events that i-th pen is blue, Now X is the number of blue pen selected is equal to the number of events A₁,A₂,…..,A_n that occur because the ith pen selected is equally likely to any of the N pens of which m are blue

and so

this gives

so the variance of hypergeometric random variable will be

in similar way for the higher moments

hence

Moments of the negative hypergeometric random variables

  consider the example of a package containing n+m vaccines of which n are special and m are ordinary, these vaccines removed one at a time, with each new removal equally likely to be any of the vaccine that remain in the package. Now let random variable Y denote the number of vaccines that need to be withdrawn until a total of r special vaccines have been removed, which is negative hypergeometric distribution, this is somehow similar with negative binomial to binomial as to hypergeometric distribution. to find the probability mass function if the kth draw gives the special vaccine after k-1 draw gives r-1 special and k-r ordinary vaccine

now the random variable Y

Y=r+X

for the events A_i

as

hence to find the variance of Y we must know the variance of X so

hence

COVARIANCE             

The relationship between two random variable can be represented by the statistical parameter covariance, before the definition of covariance of two random variable X and Y recall that the expectation of two functions g and h of random variables X and Y respectively gives

using this relation of expectation we can define covariance as

   “ The covariance between random variable X and random variable Y denoted by cov(X,Y)  is defined as

using definition of expectation and expanding we get

it is clear that if the random variables X and Y are independent then

but the converse is not true for example if

and defining the random variable Y as

so

here clearly X and Y are not independent but covariance is zero.

Properties of covariance

  Covariance between random variables X and Y has some properties as follows

using the definition off the covariance the first three properties are immediate and the fourth property follows by considering

now by definition

Variance of the sums

The important result from these properties is

as

If X_i ‘s are pairwise independent then

Example: Variance of a binomial random variable

  If X is the random variable

where X_i are the independent Bernoulli random variables such that

 then find the variance of a binomial random variable X with parameters n and p.

Solution:

since

so for single variable we have

so the variance is

Example

  For the independent random variables X_i with the respective means and variance and a new random variable with deviation as

then compute

solution:

By using the above property and definition we have

now for the random variable S

take the expectation

Example:

Find the covariance of indicator functions for the events A and B.

Solution:

for the events A and B the indicator functions are

so the expectation of these are

thus the covariance is

Example:

     Show that

where X_i are independent random variables with variance.

Solution:

The covariance using the properties and definition will be

Example:

  Calculate the mean and variance of random variable S which is the sum of n sampled values if set of N people each of whom has an opinion about a certain subject that is measured by a real number v that represents the person’s “strength of feeling” about the subject. Let  represent the strength of feeling of person  which is unknown, to collect information a sample of n from N is taken randomly, these n people are questioned and their feeling is obtained to calculate vi

Solution

let us define the indicator function as

thus we can express S as

and its expectation as

this gives the variance as

since

we have

we know the identity

so

so the mean and variance for the said random variable will be

Conclusion:

The correlation between two random variables is defined as covariance and using the covariance the sum of the variance is obtained for different random variables, the covariance and different moments with the help of definition of expectation is obtained  , if you require further reading go through

https://en.wikipedia.org/wiki/Expectation

A first course in probability by Sheldon Ross

Schaum’s Outlines of Probability and Statistics

An introduction to probability and statistics by ROHATGI and SALEH.

For more post on mathematics, please follow our Mathematics page

Mathematical Expectation and random variable    

     The mathematical expectation plays very important role in the probability theory, the basic definition and basic properties of mathematical expectation already we discussed in previous some articles now after discussing the various distributions and types of distributions, in the following article we will get familiar with some more advanced properties of mathematical expectation.

Expectation of sum of random variables | Expectation of function of random variables | Expectation of Joint probability distribution

     We know the mathematical expectation of random variable of discrete nature is

and for the continuous one is

now for the random variable X and Y if discrete then with the joint probability mass function p(x,y)

expectation of function of random variable X and Y will be

and if continuous then with the joint probability density function f(x, y) the expectation of function of random variable X and Y will be

if g is addition of these two random variables in continuous form the

and if for the random variables X and Y we have

X>Y

then the expectation also

Example

A Covid-19 hospital is uniformly distributed on the road of the length L at a point X, a vehicle carrying oxygen for the patients is at a location Y which is also uniformly distributed on the road, Find the expected distance between Covid-19 hospital and oxygen carrying vehicle if they are independent.

Solution:

To find the expected distance between X and Y we have to calculate E { | X-Y | }

Now the joint density function of X and Y will be

since

by following this we have

now the value of integral will be

Thus the expected distance between these two points will be

Expectation of Sample mean

  As the sample mean of the sequence of random variables X₁, X₂, ………, X_n with distribution function F and expected value of each as μ is

so the expectation of this sample mean will be

which shows the expected value of sample mean is also μ.

Boole’s Inequality

                Boole’s inequality can be obtained with the help of properties of expectations, suppose the random variable X defined as

where

here A_i ‘s are the random events, this means random variable X represents the occurrence of the number of events A_i and another random variable Y as

clearly

X>=Y

E[X] >= E[Y]

and so is

now if we take the value of random variable X and Y these expectation will be

and

substituting these expectation in the above inequality we will get Boole’s inequality as

Expectation of Binomial random variable | Mean of Binomial random variable

  We know that the binomial random variable is the random variable which shows number of successes in n independent trials with probability of success as p and failure as q=1-p, so if

X=X₁ + X₂+ …….+ X_n

Where

here these X_i ‘s are the Bernoulli and the expectation will be

so the expectation of X will be

Expectation of Negative binomial random variable | Mean of Negative binomial random variable

  Let a random variable X which represents the number of trials needed to collect r successes, then such a random variable is known as negative binomial random variable and it can be expressed as

here each X_i denote the number of trials required after the (i-1)st success  to obtain the total of i successes.

Since each of these X_i represent the geometric random variable and we know the expectation for the geometric random variable is

so

which is the expectation of negative binomial random variable.

Expectation of hypergeometric random variable | Mean of hypergeometric random variable

The expectation or mean of the hypergeometric random variable we will obtain with the help of a simple real life example, if n number of books are randomly selected from a shelf containing N books of which m are of mathematics, then to find the expected number of mathematics books let X denote the number of mathematics books selected then we can write X as

where

so

=n/N

which gives

which is the mean of such a hypergeometric random variable.

Expected number of matches

   This is very popular problem related to expectation, suppose that in a room there are N number of people who throw their hats in the middle of the room  and all the hats are mixed  after that each person randomly choose one hat then the expected number of people who select their own hat we can obtain by letting X to be the number of matches so

Where

since each person has equal opportunity to select any of the hat from N hats then

so

which means exactly one person on average choose his own hat.

The probability of a union of events

     Let us obtain the probability of the union of the events with the help of expectation so for the events A_i

with this we take

so the expectation of this will be

and expanding using expectation property as

since we have

and

so

this implies the probability of union as

Bounds from Expectation using Probabilistic method

    Suppose S be a finite set and f is the function on the elements of S and

here we can obtain the lower bound for this m by expectation of f(s) where “s” is any random element of S whose expectation we can calculate so

here we get expectation as the lower bound for the maximum value

Maximum-Minimum identity

 Maximum Minimum identity is the maximum of the set of numbers to the minimums of the subsets of these numbers that is for any numbers x_i

To show this let us restrict the x_i within the interval [0,1], suppose a uniform random variable U on the interval (0,1) and the events A_i as the uniform variable U is less than x_i that is

since at least one of the above event occur as U is less than one the value of x_i

and

Clearly we know

and all the events will occur if U is less than all the variables and

the probability gives

we have the result of probability of union as

following this inclusion exclusion formula for the probability

consider

this gives

since

which means

• hence we can write it as

taking expectation we can find expected values of maximum and partial minimums as

Conclusion:

The Expectation in terms of various distribution and correlation of expectation with some of the probability theory concepts were the focus of this article which shows the use of expectation as a tool to get expected values of different kind of random variables, if you require further reading go through below books.

For more articles on Mathematics, please see our Mathematics page.

https://en.wikipedia.org/wiki/Expectation

A first course in probability by Sheldon Ross

Schaum’s Outlines of Probability and Statistics

An introduction to probability and statistics by ROHATGI and SALEH

• Content
• Conditional distribution
• Discrete conditional distribution
• Example on discrete conditional distribution
• Continuous conditional distribution
• Example on Continuous conditional distribution
• Conditional distribution of bivariate normal distribution
• Joint Probability distribution of function of random variables
• Examples on Joint Probability distribution of function of random variables

Conditional distribution

   It is very interesting to discuss the conditional case of distribution when two random variables follows the distribution satisfying one given another, we first briefly see the conditional distribution in both the case of random variables, discrete and continuous then after studying some prerequisites we focus on the conditional expectations.

Discrete conditional distribution

     With the help of joint probability mass function in joint distribution we define conditional distribution for the discrete random variables X and Y using conditional probability for X given Y as the distribution with the probability mass function

provided the denominator probability is greater than zero, in similar we can write this as

in the joint probability if the X and Y are independent random variables then this will turn into

so the discrete conditional distribution or conditional distribution for the discrete random variables X given Y is the random variable with the above probability mass function in similar way for Y given X we can define.

Example on discrete conditional distribution

1. Find the probability mass function of random variable X given Y=1, if the joint probability mass function for the random variables X and Y has some values as

p(0,0)=0.4 , p(0,1)=0.2, p(1,0)= 0.1, p(1,1)=0.3

Now first of all for the value Y=1 we have

so using the definition of probability mass function

we have

and

• obtain the conditional distribution of X given X+Y=n, where X and Y are Poisson distributions with the parameters λ₁ and λ₂ and X and Y are independent random variables

Since the random variables X and Y are independent, so the conditional distribution will have probability mass function as

since the sum of Poisson random variable is again poisson so

thus the conditional distribution with above probability mass function will be conditional distribution for such Poisson distributions. The above case can be generalize for more than two random variables.

Continuous conditional distribution

   The Continuous conditional distribution of the random variable X given y already defined is the continuous distribution with the probability density function

denominator density is greater than zero, which  for the continuous density function is

thus the probability for such conditional density function is

In similar way as in discrete if X and Y are independent  in continuous then also

and hence

so we can write it as

Example on Continuous conditional distribution

1. Calculate conditional density function of random variable X given Y if the joint probability density function with the open interval (0,1) is given by

If for the random variable X given Y within (0,1) then by using the above density function we have

• Calculate the conditional probability

if the joint probability density function is given by

To find the conditional probability first we require the conditional density function so by the definition it would be

now using this density function in the probability the conditional probability is

Conditional distribution of bivariate normal distribution

  We know that the Bivariate normal distribution of the normal random variables X and Y with the respective means and variances as the parameters has the joint probability density function

so to find the conditional distribution for such a bivariate normal distribution for X given Y is defined by following the conditional density function of the continuous random variable and the above joint density function we have

By observing this we can say that this is normally distributed with the mean

and variance

in the similar way the conditional density function for Y given X already defined will be just interchanging the positions of the parameters of X with Y,

The marginal density function for X we can obtain from the above conditional density function by using the value of the constant

let us substitute in the integral

the density function will be now

since the total value of

by the definition of the probability so the density function will be now

which is nothing but the density function of random variable X with usual mean and variance as the parameters.

Joint Probability distribution of function of random variables

  So far we know the joint probability distribution of two random variables, now if we have functions of such random variables then what would be the joint probability distribution of those functions, how to calculate the density and distribution function because we have real life situations where we have functions of the random variables,

If Y₁ and Y₂ are the functions of the random variables X₁ and X₂ respectively which are jointly continuous then the joint continuous density function of these two functions will be

where Jacobian

and Y₁ =g₁ (X₁, X₂) and Y₂ =g₂ (X₁, X₂) for some functions g₁ and g₂ . Here g₁ and g₂ satisfies the conditions of the Jacobian as continuous and have continuous partial derivatives.

Now the probability for such functions of random variables will be

Examples on Joint Probability distribution of function of random variables

1. Find the joint density function of the random variables Y₁ =X₁ +X₂ and Y₂=X₁ -X₂ , where X₁ and X₂ are the jointly continuous with joint probability density function. also discuss for the different nature of distribution .

Here we first we will check Jacobian

since g₁(x₁, x₂)= x₁ + x₂  and g₂(x₁, x₂)= x₁ – x₂ so

simplifying Y₁ =X₁ +X₂ and Y₂=X₁ -X₂ , for the value of X₁ =1/2( Y₁ +Y₂ ) and X₂ = Y₁ -Y₂ ,

if these random variables are independent uniform random variables

or if these random variables are independent exponential random variables with usual parameters

or if these random variables are independent normal random variables then

• If X and Y are the independent standard normal variables as given

calculate the joint distribution for the respective polar coordinates.

We will convert by usual conversion X and Y into r and θ as

so the partial derivatives of these function will be

so the Jacobian using this functions is

if both the random variables X and Y are greater than zero then conditional joint density function is

now the conversion of cartesian coordinate to the polar coordinate using

so the probability density function for the positive values will be

for the different combinations of X and Y the density functions in similar ways are

now from the average of the above densities we can state the density function as

and the marginal density function from this joint density of polar coordinates over the interval (0, 2π)

• Find the joint density function for the function of random variables

U=X+Y and V=X/(X+Y)

where X and Y are the gamma distribution with parameters (α + λ) and (β +λ) respectively.

Using the definition of gamma distribution and joint distribution function the density function for the random variable X and Y will be

consider the given functions as

g₁ (x,y) =x+y , g₂ (x,y) =x/(x+y),

so the differentiation of these function is

now the Jacobian is

after simplifying the given equations the variables x=uv and y=u(1-v) the probability density function is

we can use the relation

• Calculate the joint probability density function for

Y₁ =X₁ +X₂+ X₃ , Y₂ =X₁– X₂ , Y₃ =X₁ – X₃

where the random variables X1 , X2, X3 are the standard normal random variables.

Now let us calculate the Jacobian by using partial derivatives of

Y₁ =X₁ +X₂+ X₃ , Y₂ =X₁– X₂ , Y₃ =X₁ – X₃

as

simplifying for variables X₁ , X₂ and X₃

X₁ = (Y₁ + Y₂ + Y₃)/3 , X₂ = (Y₁ – 2Y₂ + Y₃)/3 , X₃ = (Y₁ + Y₂ -2 Y₃)/3

we can generalize the joint density function as

so we have

for the normal variable the  joint probability density function is

hence

where the index is

compute the joint density function of Y₁ ……Y_n and marginal density function for Y_n where

and X_i are independent identically distributed exponential random variables with parameter λ.

for the random variables of the form

Y₁ =X₁ , Y₂ =X₁ + X₂ , ……, Y_n =X₁ + ……+ X_n

the Jacobian will be of the form

and hence its value is one, and the joint density function for the exponential random variable

and the values of the variable X_i ‘s will be

so the joint density function is

Now to find the marginal density function of Y_n we will integrate one by one  as

and

like wise

if we continue this process we will get

which is the marginal density function.

Conclusion:

The conditional distribution for the discrete and continuous random variable with different examples considering some of the types of these random variables discussed, where the independent random variable plays important role. In addition the  joint distribution for the function of joint continuous random variables also explained with suitable examples, if you require further reading go through below links.

For more post on Mathematics, please refer to our Mathematics Page

Wikipediahttps://en.wikipedia.org/wiki/joint_probability_distribution/” target=”_blank” rel=”noreferrer noopener” class=”rank-math-link”>Wikipedia.org

A first course in probability by Sheldon Ross

Schaum’s Outlines of Probability and Statistics

An introduction to probability and statistics by ROHATGI and SALEH

Content

• Jointly distributed random variables
• Joint distribution function | Joint Cumulative probability distribution function | joint probability mass function | joint probability density function
• Examples on Joint distribution
• Independent random variables and joint distribution
• Example of independent joint distribution
• SUMS OF INDEPENDENT RANDOM VARIABLES BY JOINT DISTRIBUTION
• sum of independent exponential random variables
• sum of independent Gamma random variables
• sum of independent exponential random variables
• Sum of independent normal random variable | sum of independent Normal distribution
• Sums of independent Poisson random variables
• Sums of independent binomial random variables

Jointly distributed random variables

     The jointly distributed random variables are the random variable more than one with probability jointly distributed for these random variables, in other words in experiments where the different outcome with their common probability is known as jointly distributed random variable or joint distribution, such type of situation occurs frequently while dealing the problems of the chances.

Joint distribution function | Joint Cumulative probability distribution function | joint probability mass function | joint probability density function

    For the random variables X and Y the distribution function or joint cumulative distribution function is

where the nature of the joint probability depends on the nature of random variables X and Y either discrete or continuous, and the individual distribution functions for X and Y can be obtained using this joint cumulative distribution function as

similarly for Y as

these individual distribution functions of X and Y are known as Marginal distribution functions when joint distribution is under consideration. These distributions are very helpful for getting the probabilities like

and in addition the joint probability mass function for the random variables X and Y is defined as

the individual probability mass or density functions for X and Y can be obtained with the help of such joint probability mass or density function like in terms of discrete random variables as

and in terms of continuous random variable the joint probability density function will be

where C is any two dimensional plane, and the joint distribution function for continuous random variable will be

the probability density function from this distribution function can be obtained by differentiating

and the marginal probability from the joint probability density function

as

and

with respect to the random variables X and Y respectively

Examples on Joint distribution

1. The joint probabilities for the random variables X and Y representing the number of mathematics and statistics books from a set of books which contains 3 mathematics, 4 statistics and 5 physics books if 3 books taken randomly

• Find the joint probability mass function for the sample of families having 15% no child, 20% 1 child, 35% 2 child and 30% 3 child if the family we choose randomly from this sample for child to be Boy or Girl?

The joint probability we will find by using the definition as

and this we can illustrate in the tabular form as follows

• Calculate the probabilities

if for the random variables X and Y the joint probability density function is given by

with the help of definition of joint probability for continuous random variable

and the given joint density function the first probability for the given range will be

in the similar way the probability

and finally

• Find the joint density function for the quotient X/Y of random variables X and Y if their joint probability density function is

To find the probability density function for the function X/Y we first find the joint distribution function then we will differentiate the obtained result,

so by the definition of joint distribution function and given probability density function we have

thus by differentiating this distribution function with respect to a we will get the density function as

where a is within zero to infinity.

Independent random variables and joint distribution

     In the joint distribution the probability for two random variable X and Y is said to be independent if

where A and B are the real sets. As already in terms of events we know that the independent random variables are the random variables whose events are independent.

Thus for any values of a and b

and the joint distribution or cumulative distribution function for the independent random variables X and Y will be

if we consider the discrete random variables X and Y then

since

similarly for the continuous random variable also

Example of independent joint distribution

1. If for a specific day in a hospital the patients entered are poisson distributed with parameter λ and probability of male patient as p and probability of female patient as (1-p) then show that the number of male patients and female patients entered in the hospital are independent poisson random variables with parameters λp and λ(1-p) ?

consider the number of male and female patients by random variable X and Y then

as X+Y are the total number of patients entered in the hospital which is poisson distributed so

as the probability of male patient is p and female patient is (1-p) so exactly from total fix number are male or female shows binomial probability as

using these two values we will get the above joint probability as

thus probability of male and female patients will be

and

which shows both of them are poisson random variables with the parameters λp and λ(1-p).

2. find the probability that a person has to wait for more than ten minutes at the meeting for a client as if each client and that person arrives between  12 to 1 pm following uniform distribution.

consider the random variables X and Y to denote the time for that person and client between 12 to 1 so the probability jointly for X and Y will be

calculate

where X,Y and Z are uniform random variable over the interval (0,1).

here the probability will be

for the uniform distribution the density function

for the given range so

SUMS OF INDEPENDENT RANDOM VARIABLES BY JOINT DISTRIBUTION

  The sum of independent variables X and Y with the probability density functions as continuous random variables, the cumulative distribution function will be

by differentiating this cumulative distribution function for the probability density function of these independent sums are

by following these two results we will see some continuous random variables and their sum as independent variables

sum of independent uniform random variables

   for the random variables X and Y uniformly distributed over the interval (0,1) the probability density function for both of these independent variable is

so for the sum X+Y we have

for any value a lies between zero and one

if we restrict a in between one and two it will be

this gives the triangular shape density function

if we generalize for the n independent uniform random variables 1 to n then their distribution function

by mathematical induction will be

sum of independent Gamma random variables

    If we have two independent gamma random variables with their usual density function

then following the density for the sum of independent gamma random variables

this shows the density function for the sum of gamma random variables which are independent

sum of independent exponential random variables

    In the similar way as gamma random variable the sum of independent exponential random variables we can obtain density function and distribution function by just specifically assigning values of gamma random variables.

Sum of independent normal random variable | sum of independent Normal distribution

                If we have n number of independent normal random variables Xi , i=1,2,3,4….n with respective means μi and variances σ2i then their sum is also normal random variable with the mean as Σμi  and variances Σσ2i

    We first show the normally distributed independent sum for two normal random variable X with the parameters 0 and σ² and Y with the parameters 0 and 1, let us find the probability density function for the sum X+Y with

in the joint distribution density function

with the help of definition of density function of normal distribution

thus the density function will be

which is nothing but the density function of a normal distribution with mean 0 and variance (1+σ2) following the same argument we can say

with usual mean and variances. If we take the expansion and observe the sum is normally distributed with the mean as the sum of the respective means and variance as the sum of the respective variances,

thus in the same way the nth sum will be the normally distributed random variable with the mean as Σμ_i  and variances Σσ²_i

Sums of independent Poisson random variables

If we have two independent Poisson random variables X and Y with parameters λ₁ and λ₂ then their sum X+Y is also Poisson random variable or Poisson distributed

since X and Y are Poisson distributed and we can write their sum as the union of disjoint events so

by using the of probability of independent random variables

so we get the sum X+Y is also Poisson distributed with the mean λ₁ +λ₂

Sums of independent binomial random variables

                If we have two independent binomial random variables X and Y with parameters (n,p) and (m, p) then their sum X+Y is also binomial random variable or Binomial distributed with parameter (n+m, p)

let use the probability of the sum with definition of binomial as

which gives

so the sum X+Y is also binomially distributed with parameter (n+m, p).

Conclusion:

The concept of jointly distributed random variables which gives the distribution comparatively for more than one variable in the situation is discussed in addition the basic concept of independent random variable with the help of joint distribution and sum of independent variables with some example of distribution is given with their parameters, if you require further reading go through mentioned books. For more post on mathematics, please click here.

https://en.wikipedia.org

A first course in probability by Sheldon Ross

Schaum’s Outlines of Probability and Statistics

An introduction to probability and statistics by ROHATGI and SALEH

Content

1. Special form of Gamma distributions and relationships of Gamma distribution
2. Gamma distribution exponential family
3. Relationship between gamma and normal distribution
4. Poisson gamma distribution | poisson gamma distribution negative binomial
5. Weibull gamma distribution
6. Application of gamma distribution in real life | gamma distribution uses | application of gamma distribution in statistics 
7. Beta gamma distribution | relationship between gamma and beta distribution
8. Bivariate gamma distribution
9. Double gamma distribution
10. Relation between gamma and exponential distribution | exponential and gamma distribution | gamma exponential distribution
11. Fit gamma distribution
12. Shifted gamma distribution
13. Truncated gamma distribution
14. Survival function of gamma distribution
15. MLE of gamma distribution | maximum likelihood gamma distribution | likelihood function of gamma distribution
16. Gamma distribution parameter estimation method of moments | method of moments estimator gamma distribution
17. Confidence interval for gamma distribution
18. Gamma distribution conjugate prior for exponential distribution | gamma prior distribution | posterior distribution poisson gamma
19. Gamma distribution quantile function
20. Generalized gamma distribution
21. Beta generalized gamma distribution

Special form of Gamma distributions and relationships of Gamma distribution

  In this article we will discuss the special forms of gamma distributions and the relationships of gamma distribution with different continuous and discrete random variables also some estimation methods  in sampling of population using gamma distribution is briefly discuss.

Gamma distribution exponential family

  The gamma distribution exponential family and it is two parameter exponential family which is largely and applicable family of distribution as most of real life problems can be modelled in the gamma distribution exponential family and the quick and useful calculation  within the exponential family can be done easily, in the two parameter if we take probability density function as

if we restrict the known value of α (alpha) this two parameter family will reduce to one parameter exponential family

and for λ (lambda)

Relationship between gamma and normal distribution

  In the probability density function of gamma distribution if we take alpha nearer to 50 we will get the nature of density function as

even the shape parameter in gamma distribution we are increasing which is resulting in similarity of normal distribution normal curve, if we tend shape parameter alpha tends to infinity the gamma distribution will be more symmetric and normal but as alpha tends to infinity value of x in gamma distribution will tends to minus infinity which result in semi infinite support of gamma distribution infinite hence even gamma distribution becomes symmetric but not same with normal distribution.

poisson gamma distribution | poisson gamma distribution negative binomial

   The poisson gamma distribution and binomial distribution are the discrete random variable whose random variable deals with the discrete values specifically success and failure in the form of Bernoulli trials which gives random success or failure as a result only, now the mixture of Poisson and gamma distribution also known as negative binomial distribution is the outcome of the repeated trial of Bernoulli’s trial, this can be parameterize in different way as if r-th success occurs in number of trials then it can be parameterize as

and if the number of failures before the r-th success then it can be parameterize as

and considering the values of r and p

the general form of the parameterization for the negative binomial or poisson gamma distribution is

and alternative one is

this binomial distribution is known as negative because of the coefficient

and this negative binomial or poisson gamma distribution is well define as the total probability we will get as one for this distribution

The mean and variance for this negative binomial or poisson gamma distribution is

the poisson and gamma relation we can get by the following calculation

Thus negative binomial is the mixture of poisson and gamma distribution and this distribution is used in day to day problems modelling where discrete and continuous mixture we require.

Weibull gamma distribution

   There are generalization of exponential distribution which involve Weibull as well as gamma distribution as the Weibull distribution has the probability density function as

and cumulative distribution function as

where as pdf and cdf of gamma distribution is already we discussed above the main connection between Weibull and gamma distribution is both are generalization of exponential distribution the difference between them is when power of variable is greater than one then Weibull distribution gives quick result while for less than 1 gamma gives quick result.

     We will not discuss here generalized Weibull gamma distribution that require separate discussion.

application of gamma distribution in real life | gamma distribution uses | application of gamma distribution in statistics 

  There are number of  application where gamma distribution is used to model the situation such as insurance claim to aggregate, rainfall amount accumulation, for any product its manufacturing and distribution, the crowd on specific web,  and in telecom exchange etc. actually the gamma distribution give the wait time prediction till next event for nth event. There are number of application of gamma distribution in real life.

beta gamma distribution | relationship between gamma and beta distribution

    The beta distribution is the random variable with the probability density function

where

which has the relationship with gamma function as

and beta distribution related to gamma distribution as if X be gamma distribution with parameter alpha and beta as one and Y be the gamma distribution with parameter alpha as one and beta then the random variable X/(X+Y) is beta distribution.

or If X is Gamma(α,1) and Y is Gamma (1, β) then the random variable X/(X+Y) is Beta (α, β) 

and also

bivariate gamma distribution

     A two dimensional or bivariate random variable is continuous if there exists a function f(x,y) such that the joint distribution function

where

and the joint probability density function obtained by

there are number of bivariate gamma distribution one of them is the bivariate gamma distribution with probability density function as

double gamma distribution

  Double gamma distribution is one of the bivariate distribution with gamma random variables having parameter alpha and one with joint probability density function as

this density forms the double gamma distribution with respective random variables and the moment generating function for double gamma distribution is

relation between gamma and exponential distribution | exponential and gamma distribution | gamma exponential distribution

   since the exponential distribution is the distribution with the probability density function

and the gamma distribution has the probability density function

clearly the value of alpha if we put as one we will get the exponential distribution, that is the gamma distribution is nothing but the generalization of the exponential distribution, which predict the wait time till the occurrence of next nth event while exponential distribution predict the wait time till the occurrence of the next event.

fit gamma distribution

   As far as fitting the given data in the form of gamma distribution imply finding the two parameter probability density function which involve shape, location and scale parameters so finding these parameters with different application and calculating the mean, variance, standard deviation and moment generating function is the fitting of gamma distribution, since different real life problems will be modelled in gamma distribution so the information as per situation must be fit in gamma distribution for this purpose various technique in various environment is already there e.g in R, Matlab, excel etc.

shifted gamma distribution

     There are as per application and need whenever the requirement of shifting the distribution required from two parameter gamma distribution the new generalized three parameter or any another generalized gamma distribution shift the shape location and scale , such gamma distribution is known as shifted gamma distribution

truncated gamma distribution

     If we restrict the range or domain of the gamma distribution for the shape scale and location parameters the restricted gamma distribution is known as truncated gamma distribution based on the conditions.

survival function of gamma distribution

                The survival function for the gamma distribution is defined the function s(x) as follows

mle of gamma distribution | maximum likelihood gamma distribution | likelihood function of gamma distribution

we know that the maximum likelihood take the sample from the population as a representative and this sample consider as an estimator for the probability density function to maximize for the parameters of density function, before going to gamma distribution recall some basics as for the random variable X the probability density function with theta as parameter has likelihood function as

this we can express as

and method of maximizing this likelihood function can be

if such theta satisfy this equation, and as log is monotone function we can write in terms of log

and such a supremum exists if

now we apply the maximum likelihood for the gamma distribution function as

the log likelihood of the function will be

so is

and hence

This can be achieved also as

by

and the parameter can be obtained by differentiating

gamma distribution parameter estimation method of moments | method of moments estimator gamma distribution

   We can calculate the moments of the population and sample with the help of expectation of nth order respectively, the method of moment equates these moments of distribution and sample to estimate the parameters, suppose we have sample of gamma random variable with the probability density function as

we know the first tow moments for this probability density function is

so

we will get from the second moment if we substitute lambda

and from this value of alpha is

and now lambda will be

and moment estimator using sample will be

confidence interval for gamma distribution

   confidence interval for gamma distribution is the way to estimate the information and its uncertainty which tells the interval is expected to have the true value of the parameter at what percent, this confidence interval is obtained from the observations of random variables, since it is obtained from random it itself is random to get the confidence interval for the gamma distribution there are different techniques in different application that we have to follow.

gamma distribution conjugate prior for exponential distribution | gamma prior distribution | posterior distribution poisson gamma

     The posterior and prior distribution  are the terminologies of Bayesian probability theory and they are conjugate to each other, any two distributions are conjugate if the posterior of one distribution is another distribution, in terms of theta let us show that gamma distribution is conjugate prior to the exponential distribution

if the probability density function of gamma distribution in terms of theta is as

assume the distribution function for theta is exponential from given data

so the joint distribution will be

and using the relation

we have

which is

so gamma distribution is conjugate prior to exponential distribution as posterior is gamma distribution.

gamma distribution quantile function

   Qauntile function of gamma distribution will be the function that gives the points in gamma distribution which relate the rank order of the values in gamma distribution, this require cumulative distribution function and for different language different algorithm and functions for the quantile of gamma distribution.

generalized gamma distribution

    As gamma distribution itself is the generalization of exponential family of distribution adding more parameters to this distribution gives us generalized gamma distribution which is the further generalization of this distribution family, the physical requirements gives different generalization one of the frequent one is using the probability density function as

the cumulative distribution function for such generalized gamma distribution can be obtained by

where the numerator represents the incomplete gamma function as

using this incomplete gamma function the survival function for the generalized gamma distribution can be obtained as

another version of this three parameter generalized gamma distribution having probability density function is

where k, β, θ are the parameters greater than zero, these generalization has convergence issues to overcome the Weibull parameters replaces

using this parameterization the convergence of the density function obtained so the more generalization for the gamma distribution with convergence is the distribution with probability density function as

Beta generalized gamma distribution

   The gamma distribution involving the parameter beta in the density function because of which sometimes gamma distribution is known as the beta generalized gamma distribution with the density function

with cumulative distribution function as

which is already discussed in detail in the discussion of gamma distribution, the further beta generalized gamma distribution is defined with the cdf as

where B(a,b) is the beta function , and the probability density function for this can be obtained by differentiation and the density function will be

here the G(x) is the above defined cumulative distribution function of gamma distribution, if we put this value then the cumulative distribution function of beta generalized gamma distribution is

and the probability density function

the remaining properties can be extended for this beta generalized gamma distribution with usual definitions.

Conclusion:

There are different forms and generalization of gamma distribution and Gamma distribution exponential family as per the real life situations so possible such forms and generalizations were covered in addition with the estimation  methods of gamma distribution in population sampling of information, if you require further reading on Gamma distribution exponential family, please go through below link and books. For more topics on Mathematics please visit our page.

https://en.wikipedia.org/wiki/Gamma_distribution

A first course in probability by Sheldon Ross

Schaum’s Outlines of Probability and Statistics

An introduction to probability and statistics by ROHATGI and SALEH

Gamma Distribution

One of the continuous random variable and continuous distribution is the Gamma distribution, As we know the continuous random variable deals with the continuous values or intervals so is the Gamma distribution with specific probability density function and probability mass function, in the successive discussion we discuss in detail the concept, properties and results with examples of gamma random variable and gamma distribution.

Gamma random variable or Gamma distribution | what is gamma distribution | define gamma distribution | gamma distribution density function | gamma distribution probability density function | gamma distribution proof

A continuous random variable with probability density function

is known to be Gamma random variable or Gamma distribution where the α>0, λ>0 and the gamma function

we have the very frequent property of gamma function by integration by parts as

If we continue the process starting from n then

and lastly the value of gamma of one will be

thus the value will be

cdf of gamma distribution | cumulative gamma distribution | integration of gamma distribution

The cumulative distribution function(cdf) of gamma random variable or simply the distribution function of gamma random variable is same as that of continuous random variable provided the probability density function is different i.e

here the probability density function is as defined above for the gamma distribution, the cumulative distribution function we can write also as

in both of the above formats the value of pdf is as follows

where the α >0, λ>0 are real numbers.

Gamma distribution formula | formula for gamma distribution | gamma distribution equation | gamma distribution derivation

To find the probability for the gamma random variable the probability density function we have to use for different given α >0 , λ >0 is as

and using the above pdf the distribution for the gamma random variable we can obtain by

Thus the gamma distribution formula require the pdf value and the limits for the gamma random variable as per the requirement.

Gamma distribution example

show that the total probability for the gamma distribution is one with the given probability density function i.e

for λ >0, α>0.
Solution:
using the formula for the gamma distribution

since the probability density function for the gamma distribution is

which is zero for all the value less than zero so the probability will be now

using the definition of gamma function

and substitution we get

thus

Gamma distribution mean and variance | expectation and variance of gamma distribution | expected value and variance of gamma distribution | Mean of gamma distribution | expected value of gamma distribution | expectation of gamma distribution

In the following discussion we will find the mean and variance for the gamma distribution with the help of standard definitions of expectation and variance of continuous random variables,

The expected value or mean of the continuous random variable X with probability density function

or Gamma random variable X will be

mean of gamma distribution proof | expected value of gamma distribution proof

To obtain the expected value or mean of gamma distribution we will follow the gamma function definition and property,
first by the definition of expectation of continuous random variable and probability density function of gamma random variable we have

by cancelling the common factor and using the definition of gamma function

now as we have the property of gamma function

the value of expectation will be

thus the mean or expected value of gamma random variable or gamma distribution we get is

variance of gamma distribution | variance of a gamma distribution

The variance for the gamma random variable with the given probability density function

or variance of the gamma distribution will be

variance of gamma distribution proof

As we know that the variance is the difference of the expected values as

for the gamma distribution we already have the value of mean

now first let us calculate the value of E[X²], so by definition of expectation for the continuous random variable we have
since the function f(x) is the probability distribution function of gamma distribution as

so the integral will be from zero to infinity only

so by definition of the gamma function we can write

Thus using the property of the gamma function we got the value as

Now putting the value of these expectation in

thus, the value of variance of gamma distribution or gamma random variable is

Gamma distribution parameters | two parameter gamma distribution | 2 variable gamma distribution

The Gamma distribution with the parameters λ>0, α>0 and the probability density function

has statistical parameters mean and variance as

and

since λ is positive real number, to simplify and easy handling another way is to set λ=1/β so this gives the probability density function in the form

in brief the distribution function or cumulative distribution function for this density we can express as

this gamma density function gives the mean and variance as

and

which is obvious by the substitution.
Both the way are commonly used either the gamma distribution with the parameter α and λ denoted by gamma (α, λ) or the gamma distribution with the parameters β and λ denoted by gamma (β, λ) with the respective statistical parameters mean and variance in each of the form.
Both are nothing but the same.

Gamma distribution plot | gamma distribution graph| gamma distribution histogram

The nature of the gamma distribution we can easily visualize with the help of graph for some of specific values of the parameters, here we draw the plots for the probability density function and cumulative density function for some values of parameters
let us take probability density function as

then cumulative distribution function will be

Description: graphs for the probability density function and cumulative distribution function by fixing the value of alpha as 1 and varying the value of beta.

Description: graphs for the probability density function and cumulative distribution function by fixing the value of alpha as 2 and varying the value of beta

Description: graphs for the probability density function and cumulative distribution function by fixing the value of alpha as 3 and varying the value of beta

Description: graphs for the probability density function and cumulative distribution function by fixing the value of beta  as 1 and varying the value of alpha

Description: graphs for the probability density function and cumulative distribution function by fixing the value of beta  as 2 and varying the value of alpha

Description: graphs for the probability density function and cumulative distribution function by fixing the value of beta as 3 and varying the value of alpha.

In general different curves as for alpha varying is

Gamma distribution table | standard gamma distribution table

The numerical value of gamma function

known as incomplete gamma function numerical values as follows

The gamma distribution numerical value for sketching the plot for the probability density function and cumulative distribution function for some initial values are as follows

1x	f(x),α=1,β=1	f(x),α=2,β=2	f(x),α=3,β=3	P(x),α=1,β=1	P(x),α=2,β=2	P(x),α=3,β=3
0	1	0	0	0	0	0
0.1	0.904837418	0.02378073561	1.791140927E-4	0.09516258196	0.001209104274	6.020557215E-6
0.2	0.8187307531	0.0452418709	6.929681371E-4	0.1812692469	0.00467884016	4.697822176E-5
0.3	0.7408182207	0.06455309823	0.001508062363	0.2591817793	0.01018582711	1.546530703E-4
0.4	0.670320046	0.08187307531	0.00259310613	0.329679954	0.01752309631	3.575866931E-4
0.5	0.6065306597	0.09735009788	0.003918896875	0.3934693403	0.02649902116	6.812970042E-4
0.6	0.5488116361	0.1111227331	0.005458205021	0.4511883639	0.03693631311	0.001148481245
0.7	0.4965853038	0.1233204157	0.007185664583	0.5034146962	0.04867107888	0.001779207768
0.8	0.4493289641	0.1340640092	0.009077669195	0.5506710359	0.06155193555	0.002591097152
0.9	0.4065696597	0.1434663341	0.01111227331	0.5934303403	0.07543918015	0.003599493183
1	0.3678794412	0.1516326649	0.01326909834	0.6321205588	0.09020401043	0.004817624203
1.1	0.3328710837	0.1586611979	0.01552924352	0.6671289163	0.1057277939	0.006256755309
1.2	0.3011942119	0.1646434908	0.01787520123	0.6988057881	0.1219013822	0.007926331867
1.3	0.272531793	0.1696648775	0.0202907766	0.727468207	0.1386244683	0.00983411477
1.4	0.2465969639	0.1738048563	0.02276101124	0.7534030361	0.1558049836	0.01198630787
1.5	0.2231301601	0.1771374573	0.02527211082	0.7768698399	0.1733585327	0.01438767797
1.6	0.201896518	0.1797315857	0.02781137633	0.798103482	0.1912078646	0.01704166775
1.7	0.1826835241	0.1816513461	0.03036713894	0.8173164759	0.2092823759	0.01995050206
1.8	0.1652988882	0.1829563469	0.03292869817	0.8347011118	0.2275176465	0.02311528775
1.9	0.1495686192	0.1837019861	0.03548626327	0.8504313808	0.2458550043	0.02653610761
2	0.1353352832	0.1839397206	0.03803089771	0.8646647168	0.2642411177	0.03021210849
2.1	0.1224564283	0.1837173183	0.04055446648	0.8775435717	0.2826276143	0.03414158413
2.2	0.1108031584	0.183079096	0.04304958625	0.8891968416	0.3009707242	0.03832205271
2.3	0.1002588437	0.1820661424	0.04550957811	0.8997411563	0.3192309458	0.04275032971
2.4	0.09071795329	0.1807165272	0.04792842284	0.9092820467	0.3373727338	0.04742259607
2.5	0.08208499862	0.179065498	0.05030071858	0.9179150014	0.3553642071	0.052334462
2.6	0.07427357821	0.1771456655	0.05262164073	0.9257264218	0.373176876	0.05748102674
2.7	0.06720551274	0.1749871759	0.05488690407	0.9327944873	0.3907853875	0.0628569343
2.8	0.06081006263	0.1726178748	0.05709272688	0.9391899374	0.4081672865	0.06845642568
2.9	0.05502322006	0.1700634589	0.05923579709	0.9449767799	0.4253027942	0.07427338744
3	0.04978706837	0.1673476201	0.0613132402	0.9502129316	0.4421745996	0.08030139707

finding alpha and beta for gamma distribution | how to calculate alpha and beta for gamma distribution | gamma distribution parameter estimation

For a gamma distribution finding alpha and beta we will take mean and variance of the gamma distribution

and

now we will get value of beta as

so

and

thus

only taking some fractions from the gamma distribution we will get the value of alpha and beta.

gamma distribution problems and solutions | gamma distribution example problems | gamma distribution tutorial | gamma distribution question

1. Consider the time require to resolve the problem for a customer is gamma distributed in hours with the mean 1.5 and variance 0.75 what would be the probability that the problem resolving time exceed 2 hours, if time exceeds 2 hours what would be the probability that the problem will resolved in at least 5 hours.

solution: since the random variable is gamma distributed with mean 1.5 and variance 0.75 so we can find the values of alpha and beta and with the help of these values the probability will be

P(X>2)=13e^-4=0.2381

and

P(X>5 | X>2)=(61/13)e^-6=0.011631

2. If the negative feedback in week from the users is modelled in gamma distribution with parameters alpha 2 and beta as 4 after the 12 week negative feedback came after restructuring the quality, from this information can restructuring improves the performance ?

solution: As this is modelled in gamma distribution with α=2, β=4

we will find the mean and standard deviation as μ =E(x)=α * β=4 * 2=8

since the value X=12 is within the standard deviation from the mean so we can not say this is improvement or not by the restructuring the quality, to prove the improvement caused by the restructuring information given is insufficient.

3. Let X be the gamma distribution with parameters α=1/2, λ=1/2 , find the probability density function for the function Y=Square root of X

Solution: let us calculate the cumulative distribution function for Y as

now differentiating this with respect to y gives the probability density function for Y as

and the range for y will be from 0 to infinity

Conclusion:

The concept of gamma distribution in probability and statistic is the one of the important day to day applicable distribution of exponential family, all the basic to higher level concept were discussed so far related to gamma distribution, if you require further reading, please go through mentioned books. You can also visit out mathematics page for more Topic

https://en.wikipedia.org/wiki/Gamma_distribution
A first course in probability by Sheldon Ross
Schaum’s Outlines of Probability and Statistics
An introduction to probability and statistics by ROHATGI and SALEH