Moment Generating Functions: 13 Important Facts

Moment generating function    

Moment generating function is very important function which generates the moments of random variable which involve mean, standard deviation and variance etc., so with the help of moment generating function only, we can find basic moments as well as higher moments, In this article we will see moment generating functions for the different discrete and continuous random variables. since the Moment generating function(MGF) is defined with the help of mathematical expectation denoted by M(t) as

and using the definition of expectation for the discrete and continuous random variable this function will be

which by substituting the value of t as zero generates respective moments. These moments we have to collect by differentiating this moment generating function for example for first moment or mean we can obtain by differentiating once as

This gives the hint that differentiation is interchangeable under the expectation and we can write it as

and

if t=0 the above moments will be

and

In general we can say that

hence

Moment generating function of Binomial distribution||Binomial distribution moment generating function||MGF of Binomial distribution||Mean and Variance of Binomial distribution using moment generating function

The Moment generating function for the random variable X which is Binomially distribution will follow the probability function of binomial distribution with the parameters n and p as

which is the result by binomial theorem, now differentiating and putting the value of t=0

which is the mean or first moment of binomial distribution similarly the second moment will be

so the variance of the binomial distribution will be

which is the standard mean and variance of Binomial distribution, similarly the higher moments also we can find using this moment generating function.

Moment generating function of Poisson distribution||Poisson distribution moment generating function||MGF of Poisson distribution||Mean and Variance of Poisson distribution using moment generating function

 If we have the random variable X which is Poisson distributed with parameter Lambda then the moment generating function for this distribution will be

now differentiating this will give

this gives

which gives the mean and variance for the Poisson distribution same which is true

Moment generating function of Exponential distribution||Exponential distribution moment generating function||MGF of Exponential distribution||Mean and Variance of Exponential distribution using moment generating function

                The Moment generating function for the exponential random variable X by following the definition is

here the value of t is less than the parameter lambda, now differentiating this will give

which provides the moments

clearly

Which are the mean and variance of exponential distribution.

Moment generating function of Normal distribution||Normal distribution moment generating function||MGF of Normal distribution||Mean and Variance of Normal distribution using moment generating function

  The Moment generating function for the continuous distributions also same as the discrete one so the moment generating function for the normal distribution with standard probability density function will be

this integration we can solve by adjustment as

since the value of integration is 1. Thus the moment generating function for the standard normal variate will be

from this we can find for any general normal random variable the moment generating function by using the relation

thus

so differetiation gives us

thus

so the variance will be

Moment generating function of Sum of random variables

The Moment generating function of sum of random variables gives important property that it equals the product of moment generating function of respective independent random variables that is for independent random variables X and Y then the moment generating function for the sum of random variable X+Y is

here moment generating functions of each X and Y are independent by the property of mathematical expectation. In the succession we will find the sum of moment generating functions of different distributions.

Sum of Binomial random variables

If the random variables X and Y are distributed by Binomial distribution with the parameters (n,p) and (m,p) respectively then moment generating function of their sum X+Y will be

where the parameters for the sum is (n+m,p).

Sum of Poisson random variables

The distribution for the sum of independent random variables X and Y with respective means which are distributed by Poisson distribution we can find as

Where

is the mean of Poisson random variable X+Y.

Sum of Normal random variables

     Consider the independent normal random variables X and Y with the parameters

then for the sum of random variables X+Y with parameters

so the moment generating function will be

which is moment generating function with additive mean and variance.

Sum of random number of random variables

To find the moment generating function of the sum of random number of random variables let us assume the random variable

where the random variables X₁,X₂, … are sequence of random variables of any type, which are independent and identically distributed then the moment generating function will be

Which gives the moment generating function of Y on differentiation as

hence

in the similar way the differentiation two times will give

which give

thus the variance will be

Example of Chi-square random variable

Calculate the moment generating function of the Chi-squared random variable with n-degree of freedom.

Solution: consider the Chi-squared random variable with the n-degree of freedom for

the sequence of standard normal variables then the moment generating function will be

so it gives

the normal density with mean 0 and variance σ² integrates to 1

which is the required moment generating function of n degree of freedom.

Example of Uniform random variable

Find the moment generating function of random variable X which is binomially distributed with parameters n and p given the conditional random variable Y=p on the interval (0,1)

Solution: To find the moment generating function of random variable X given Y

using the binomial distribution, sin Y is the Uniform random variable on the interval (0,1)

Joint moment generating function

The Joint moment generating function for the n number of random variables X₁,X₂,…,X_n

where t₁,t₂,……t_n are the real numbers, from the joint moment generating function we can find the individual moment generating function as

Theorem: The random variables X₁,X₂,…,X_n are independent if and only if the joint mement generating function

Proof: Let us assume that the given random variables X₁,X₂,…,X_n are independent then

Now assume that the joint moment generating function satisfies the equation

• to prove the random variables X₁,X₂,…,X_n are independent we have the result that the joint moment generating function uniquely gives the joint distribution(this is another important result which requires proof) so we must have joint distribution which shows the random variables are independent, hence the necessary and sufficient condition proved.

Example of Joint Moment generating function

1.Calculate the joint moment generating function of the random variable X+Y and X-Y

Solution : Since the sum of random variables X+Y and subtraction of random variables X-Y are independent as for the independent random variables X and Y so the joint moment generating function for these will be

as this moment generating function determine the joint distribution so from this we can have X+Y and X-Y are independent random variables.

2. Consider for the experiment the number of events counted and uncounted distributed by poisson distribution with probability p and the mean λ, show that the number of counted and uncounted events are independent with respective means λp and λ(1-p).

Solution: We will consider X as the number of events and X_c the number of counted events so the number of uncounted events is X-X_c,the joint moment genrating function will generate moment

and by the moment generating function of binomial distribution

and taking expectation off these will give

Conclusion:

By using the standard definition of moment generating function the moments for the different distributions like binomial, poisson, normal etc were discussed and the sum of these random variables either the discrete or continuous the moment generating function for those and joint moment generating function were obtained with suitable examples , if you require further reading go through below books.

For more articles on Mathematics, please see our Mathematics page.

A first course in probability by Sheldon Ross

Schaum’s Outlines of Probability and Statistics

An introduction to probability and statistics by ROHATGI and SALEH