So one more thing that is a conditional expectation.
So since I said X given Y is a random variable, I can go for finding out what is the expectation
of X given Y. So this is called the conditional expectation. That means the X given Y is the,
still it is a random variable, but it is a conditional distribution. Therefore, finding
out the expectation for that, that is called the conditional expectation.
Suppose I treat both the random variable are continuous case, then the conditional expectation
is nothing, but minus infinity to infinity x times f x given y of x given y integration
with respect to x. That means by treating X and Y are continuous random variable, I
can able to define the conditional expectation is this provided this expectation exist. That
means in absolute sense if this integration converges.
Then without absolute whatever the value we are going to get that is going to the conditional
expectation of the random variable. And if you note that since the Y also can take any
value, therefore this is a function of y. Not only this is a function of y, the conditional
expectation is a random variable also. That means X given Y is a random variable, the
expectation of X given Y is a function of y and Y is a random variable. It takes a different
values y.
Therefore, expectation of X given Y is also a random variable. That means you can able
to find out what is the expectation of, expectation X given Y. If you compute that it is going
to be expectation of X. This is a very important property in which you are relating two different
random variables with the conditional sense and if you are trying to find out the expectation
of that, that is going to be the original expectation.
That means the usage of this concept instead of finding out the expectation of one random
variable, if it is easy to find out the conditional expectation then you find out the expectation
of conditional expectation that is same as the original expectation. Suppose you have
two random variables or independent random variables, then you know that there is no
dependency over the random variable X and Y.
Therefore, the expectation of Xgiven Y that is same as the expectation of X. So this can
be validated here also because this expectation of X given Y is going to be expectation of
X is a constant and the expectation of a constant is a constant that is same as the same constant,
so that can be cross checked. So here I have given expectation of X given Y in the integration
form, if both the random variables are continuous.
Then accordingly you have to use initially the joint probability mass function then conditional
probability mass function to get that conditional expectation and this conditional expectation
is very much important to give one important property called Martingale property in the
stochastic process, in which you are going to discuss not only two random variables,
you are going to discuss, you have n random variables.
And you can try to find out what is the conditional expectation of one random variable, given
that other random variable takes some value already. So there we are going to find out
what is the conditional expectation of n dimensional random variable with given that remaining
n minus 1 random variable take already some value. So here I have given only with the
two random variables how to compute the conditional expectations.
But as such you are going to find out the conditional expectation of n random variables
with n minus 1 random variables already taken some values.
So before I go to another concept, let me just give a few examples in which I have already
given if both the random variables are of a discrete type, I have given example of joint
probability mass function as 1 divided by 2 power x plus y and x takes a value 1, 2
and so on and y takes a value 1, 2 and so on. So this is the joint probability mass
function example.
Suppose you have random variables of the continuous type, then I can give one simple example of
the joint probability density function of two dimensional continuous type random variable
as joint probability density function, lambda times mu e power minus lambda x minus mu y,
where x can take the value greater than 0, y can take the value greater than 0, and lambda
is strictly greater than 0 as well as mu greater than 0.
So this is going to be the joint probability density function of a two dimensional continuous
random variable. You can cross check this is going to be joint because it is going to
be always take greater than or equal to 0 values for all x and y and if you make a double
integration over minus to infinity over x and y, then that is going to be 1. And you
can verify the other one.
If you find out the marginal distribution of this random variable, you may land up the
marginal distribution of this random variable is going to be lambda times e power minus
lambda x and similarly if you find out the marginal distribution of the same, one, you
will get mu times minus mu y, and if you cross check the product is going to be the joint
probability density function, then you can conclude both the random variables are independent
random variable.
Similarly, you can find out what is the marginal distribution of the random variable X, similarly
marginal distribution of Y, if you cross check the similar property of independent, then
that is satisfied, therefore, you can conclude here the random variables X and Y both are
discrete as well as both are independent random variable also. So the advantage with the independent
random variable, always from the joint you can find out the marginals.
But if you have marginals, you cannot find out the joints unless otherwise they are the
independent random variable. Therefore, the independent random variables makes easier
to find out the joint distribution with the provided marginal distribution. And here is
the one simple example of bivariate normal distribution.
In which, both the random variables X and Y are normally distributed, therefore the
together joint distribution is going to be of the form.
Let me write the joint probability density function of two dimensional normal distribution
random variable as 1 divided by 2pi, sigma 1, sigma 2, multiplied by square root of 1
minus rho square into e power minus half times 1 minus rho square multiplied by x minus mu
1 by sigma 1 whole square minus 2 times rho into x minus mu 1 by sigma 1 that is multiplied
by y minus mu 2 by sigma 2 plus y minus mu 2 by sigma 2 whole square.
So here if you find the marginal distribution of the random variable X and marginal distribution
of Y, you can conclude X is going to be normally distributed with the mean mu 1 and the variance
sigma 1 square and similarly you can come to the conclusion Y is also normally distributed
with the mean mu 2 and the variance sigma 2 square, that means if you make the plot
for the joint probability density function, that will be of this shape.
One is the X and one is the Y and this is going to be the joint probability density
function for fixed values of mu 1, mu 2 and sigma 1 and sigma 2 and this is going to be
the joint probability density function and here rho is nothing but the correlation coefficient.
That means what is the way the random variable X and Y are correlated that comes into the
picture when you are giving the joint probability density function of this random variable.
And they are not independent random variable, unless otherwise the rho is going to be zero.
So if the rho is going to be zero, then its gets simplified and you can able to verify
the joint probability density function will be the product of two probability density
function and each one is going to be a probability density of a normal distribution with a mean
mu 1 and the variance sigma 1 square and mu 2 and sigma 2 square.
So this bivariate normal distribution is very important one, when you discuss the multi
nominal normal distribution. So only we can able to give the joint probability density
function of the bivariate, so the multi-variate you can able to visualize how the joint probability
density function will look like and what is the way the other factors will come into the
picture.
So other than covariance correlation and correlation coefficient, we need the other called covariance
matrix also.
Because in the stochastic process, we are going to consider n dimensional random variable
as well as the sequence of random variables, so you should know how to define the covariance
matrix of n dimensional random variable. That means, if suppose you have a n random variables
X1 to Xn, then you can define the covariance matrix as, you just make rows X1 to Xn and
column also you make X1 to Xn, now you can fill up.
This is going to be n x n matrix in which each entity is going to be covariance of,
so that means the matrix entity of i, j is nothing but what is the covariance of that
random variable Xi with Xj. You know that the way I have given the definition covariance
of Xi and Xj, if i and j are same, then that is nothing but e of X square minus e X whole
square. Therefore, that is nothing but the variance of that random variable.
Therefore, this is going to be variance of X1 and this is going to be the variance of
X2. Therefore, all the diagonal elements are going to be variance of Xi. Whereas other
than the diagonal elements, we can fill it up this is going to be a covariance of X1
with X2 and like that the last element will be covariance of X1 with Xn. Similarly, second
row first column will be covariance of X2 with X1.
You can use the other property the covariance of Xi, Xj is same as covariance of Xj with
Xi also. Because you are trying to find out expectation of X into Y minus expectation
of X into expectation of Y. Therefore, both the covariance of X2 with X1 is same X1 with
X2. So it is going to be a, whatever the value you are going to get, it is going to be the
symmetric matrix and all the diagonal elements are going to be the variance.
So the way I have given the two dimensional normal distribution, that is a bivariate normal,
suppose you have n dimensional random vector, in which each random variable is a normal
distribution, then you need what is the covariance matrix for that, then only you can able to
write what is a joint probability density function of n dimensional random variable.