Covariance, Variance Of Sums: 7 Important Facts

COVARIANCE, VARIANCE OF SUMS, AND CORRELATIONS OF RANDOM VARIABLES

  The statistical parameters of the random variables of different nature using the definition of expectation of random variable is easy to obtain and understand, in the following we will find some parameters with the help of mathematical expectation of random variable.

Moments of the number of events that occur

    So far we know that expectation of different powers of random variable is the moments of random variables and how to find the expectation of random variable from the events if number of event occurred already, now we are interested in the expectation if pair of number of events already occurred, now if X represents the number of event occurred then for the events A₁, A₂, ….,A_n define the indicator variable I_i as

the expectation of X in discrete sense will be

because the random variable X is

now to find expectation if number of pair of event occurred already we have to use combination as

this gives expectation as

from this we get the expectation of x square and the value of variance also by

By using this discussion we focus different kinds of random variable to find such moments.

Moments of binomial random variables

   If p is the probability of success from n independent trials then lets denote A_i for the trial i as success so

and hence the variance of binomial random variable will be

because

if we generalize for k events

this expectation we can obtain successively for the value of k greater than 3  let us find for 3

using this iteration we can get

Moments of hypergeometric random variables

  The moments of this random variable we will understand with the help of an example suppose n pens are randomly selected from a box containing N pens of which m are blue, Let A_i denote the events that i-th pen is blue, Now X is the number of blue pen selected is equal to the number of events A₁,A₂,…..,A_n that occur because the ith pen selected is equally likely to any of the N pens of which m are blue

and so

this gives

so the variance of hypergeometric random variable will be

in similar way for the higher moments

hence

Moments of the negative hypergeometric random variables

  consider the example of a package containing n+m vaccines of which n are special and m are ordinary, these vaccines removed one at a time, with each new removal equally likely to be any of the vaccine that remain in the package. Now let random variable Y denote the number of vaccines that need to be withdrawn until a total of r special vaccines have been removed, which is negative hypergeometric distribution, this is somehow similar with negative binomial to binomial as to hypergeometric distribution. to find the probability mass function if the kth draw gives the special vaccine after k-1 draw gives r-1 special and k-r ordinary vaccine

now the random variable Y

Y=r+X

for the events A_i

as

hence to find the variance of Y we must know the variance of X so

hence

COVARIANCE             

The relationship between two random variable can be represented by the statistical parameter covariance, before the definition of covariance of two random variable X and Y recall that the expectation of two functions g and h of random variables X and Y respectively gives

using this relation of expectation we can define covariance as

   “ The covariance between random variable X and random variable Y denoted by cov(X,Y)  is defined as

using definition of expectation and expanding we get

it is clear that if the random variables X and Y are independent then

but the converse is not true for example if

and defining the random variable Y as

so

here clearly X and Y are not independent but covariance is zero.

Properties of covariance

  Covariance between random variables X and Y has some properties as follows

using the definition off the covariance the first three properties are immediate and the fourth property follows by considering

now by definition

Variance of the sums

The important result from these properties is

as

If X_i ‘s are pairwise independent then

Example: Variance of a binomial random variable

  If X is the random variable

where X_i are the independent Bernoulli random variables such that

 then find the variance of a binomial random variable X with parameters n and p.

Solution:

since

so for single variable we have

so the variance is

Example

  For the independent random variables X_i with the respective means and variance and a new random variable with deviation as

then compute

solution:

By using the above property and definition we have

now for the random variable S

take the expectation

Example:

Find the covariance of indicator functions for the events A and B.

Solution:

for the events A and B the indicator functions are

so the expectation of these are

thus the covariance is

Example:

     Show that

where X_i are independent random variables with variance.

Solution:

The covariance using the properties and definition will be

Example:

  Calculate the mean and variance of random variable S which is the sum of n sampled values if set of N people each of whom has an opinion about a certain subject that is measured by a real number v that represents the person’s “strength of feeling” about the subject. Let  represent the strength of feeling of person  which is unknown, to collect information a sample of n from N is taken randomly, these n people are questioned and their feeling is obtained to calculate vi

Solution

let us define the indicator function as

thus we can express S as

and its expectation as

this gives the variance as

since

we have

we know the identity

so

so the mean and variance for the said random variable will be

Conclusion:

The correlation between two random variables is defined as covariance and using the covariance the sum of the variance is obtained for different random variables, the covariance and different moments with the help of definition of expectation is obtained  , if you require further reading go through

https://en.wikipedia.org/wiki/Expectation

A first course in probability by Sheldon Ross

Schaum’s Outlines of Probability and Statistics

An introduction to probability and statistics by ROHATGI and SALEH.

For more post on mathematics, please follow our Mathematics page