WEBVTT
Kind: captions
Language: en
00:00:17.689 --> 00:00:29.469
Welcome students to lecture number 8 on the
MOOC's course on statistical inference .
00:00:29.469 --> 00:00:42.799
In the last lecture ,
we found that expectation of X bar is equal
00:00:42.799 --> 00:01:11.180
to mu where x bar is the sample mean , where
the sample X one up to xn are from normal
00:01:11.180 --> 00:01:40.070
mu sigma square . Thus, sample mean is unbiased
estimator of population mean .
00:01:40.070 --> 00:01:54.890
This we have seen ; obviously, the next question
is
00:01:54.890 --> 00:02:13.250
what is an unbiased estimator for sigma square
?
00:02:13.250 --> 00:02:22.500
That is, we have taken sample x 1 x 2 x n
and based on that we want to find an unbiased
00:02:22.500 --> 00:02:37.970
estimate for the population variance sigma
square . A natural guess will be
00:02:37.970 --> 00:03:03.730
sample variance . So, x 1 x 2 xn are my sample,
x bar is the sample mean, therefore, sigma
00:03:03.730 --> 00:03:18.570
x i minus x bar whole square upon n that is
the sample variance . The question is E is
00:03:18.570 --> 00:03:27.790
the expectation of small square is equal to
sigma square . Or in other words whether the
00:03:27.790 --> 00:03:36.690
sample variance is an unbiased estimator for
population variance .
00:03:36.690 --> 00:03:48.739
In order to find the answer, we have to compute
the expected value of small square .
00:03:48.739 --> 00:03:58.989
The expected value of small square is equal
to expected value of 1 by n sigma i is equal
00:03:58.989 --> 00:04:16.660
to 1 to n x i minus x bar whole square ; is
equal to expected value of 1 by n summation
00:04:16.660 --> 00:04:37.180
i is equal to 1 to n x i square minus 2 x
i x bar plus x bar square , is equal to expected
00:04:37.180 --> 00:05:05.350
value of 1 by n into sigma x i square 1 to
n minus 1 by n 2 x bar sigma x i plus 1 by
00:05:05.350 --> 00:05:20.860
n sigma x bar square 1 to n . This is is equal
to therefore, expected value of this whole
00:05:20.860 --> 00:05:27.240
thing .
So, this is is equal to expected value of
00:05:27.240 --> 00:05:47.580
1 by n sigma x i square minus expected value
of 1 by n sigma x i is equal to x bar. Therefore,
00:05:47.580 --> 00:06:09.060
it is 2 x bar square plus 1 by n plus expected
value of 1 by n into n x bar square .
00:06:09.060 --> 00:06:22.350
Is equal to expected value of 1 by n sigma
x i square minus 2 times expected value of
00:06:22.350 --> 00:06:37.750
x bar square plus expected value of x bar
square; is equal to expected value of 1 by
00:06:37.750 --> 00:06:44.480
n sigma x i square minus expected value of
x bar square .
00:06:44.480 --> 00:07:00.930
Now, we know that
00:07:00.930 --> 00:07:18.740
variance of x i is equal to expected value
of x i minus expected value of x i whole square
00:07:18.740 --> 00:07:49.570
. So, variants of each x i is equal to expected
value of x i square minus expected value of
00:07:49.570 --> 00:08:01.960
x i whole square, is equal to expected value
of x i square minus mu squared . And variance
00:08:01.960 --> 00:08:14.310
of x i is equal to sigma square . Therefore,
the expected value of x i square is equal
00:08:14.310 --> 00:08:33.630
to sigma square plus mu square .
By a similar logic
00:08:33.630 --> 00:08:44.670
variance of x bar is equal to expected value
of x bar square minus expected value of x
00:08:44.670 --> 00:08:56.370
bar whole square .
And we know that variance of x bar is equal
00:08:56.370 --> 00:09:12.010
to sigma square by n . Therefore, sigma square
by n is equal to expected value of x bar square
00:09:12.010 --> 00:09:24.370
minus mu square . Therefore, expected value
of x bar square is equal to sigma square by
00:09:24.370 --> 00:09:43.070
n plus mu square . So, we find that one expected
value of x i square is equal to sigma square
00:09:43.070 --> 00:09:54.800
plus mu square 2 the expected value of x bar
square is equal to sigma square by n plus
00:09:54.800 --> 00:10:02.000
mu square .
Now, we have already found that expectation
00:10:02.000 --> 00:10:41.050
of small square is equal to this.
Therefore by putting these values 1 by n summation
00:10:41.050 --> 00:11:02.050
over 1 to n sigma square plus mu square minus
sigma square by n plus mu square , is equal
00:11:02.050 --> 00:11:19.269
to sigma square plus mu square minus sigma
square by n minus mu square, is equal to sigma
00:11:19.269 --> 00:11:31.010
square into 1 minus 1 upon n .
Thus we find that expectation of small square
00:11:31.010 --> 00:11:41.210
is equal to sigma square into 1 minus 1 upon
n is equal to sigma square into n minus 1
00:11:41.210 --> 00:12:00.709
upon n. Therefore, small square is not unbiased
for sigma square .
00:12:00.709 --> 00:12:47.290
Therefore, if I ask you what is an unbiased
estimator for sigma square ,
00:12:47.290 --> 00:12:55.399
small square is equal to 1 by n into sigma
x i minus x bar whole square, i is equal to
00:12:55.399 --> 00:13:07.400
1 to n is equal to sigma x i minus x bar whole
square divided by n minus 1 .
00:13:07.400 --> 00:13:16.950
So, that shows that our logical guess that
sample variance is going to be an unbiased
00:13:16.950 --> 00:13:24.940
estimator for sigma square is not correct,
the unbiased estimator for sigma square is
00:13:24.940 --> 00:13:34.589
this quantity which is sigma x i minus x bar
whole square upon n minus 1 and we often denoted
00:13:34.589 --> 00:13:58.410
by s square .
Now, we want to find the distribution of or
00:13:58.410 --> 00:14:06.790
we can say we want to find the sampling distribution
of capital S square .
00:14:06.790 --> 00:14:14.360
In order to do that we first need some mathematical
tricks .
00:14:14.360 --> 00:14:33.870
We have seen that
when we transform
00:14:33.870 --> 00:14:55.709
from the XY plane to the ZW plane such that
Z is equal to G 1 of X Y, and W is equal to
00:14:55.709 --> 00:15:37.450
G 2 of X Y we can get joint pdf of z w as
the joint distribution of XY expressed in
00:15:37.450 --> 00:15:49.610
zw multiplied by the Jacobean where the Jacobean
is the determinant of del x, del z, del x,
00:15:49.610 --> 00:16:10.050
del w, del y, del z and del y del w .
Now, suppose we want to extend it to n dimension
00:16:10.050 --> 00:16:23.800
. The most important concept was that given
z w we can use inverse transformation to get
00:16:23.800 --> 00:16:51.690
unique solution for x y .
Now, we want to extend to n dimension . Suppose
00:16:51.690 --> 00:17:11.220
we write y 1 y 2 y n as a function of
x 1 x 2 x n . So, we have n random variables
00:17:11.220 --> 00:17:21.519
x 1 x 2 x n, we want to make it transformation
to new set of variables y 1 y 2 y n using
00:17:21.519 --> 00:17:39.820
A transformation function g .
In particular, , let us consider g to be linear
00:17:39.820 --> 00:18:01.899
transformation ;
that is, y 1 y 2 yn is equal to a times x
00:18:01.899 --> 00:18:32.049
1 x 2 xn where A is equal to a 1 1, a 1 2,
a 1 3, a 1 n , a n 1, a n 2, a n n .
00:18:32.049 --> 00:18:50.809
We can get
x 1 x 2 xn uniquely from y 1 y 2 yn . Let
00:18:50.809 --> 00:19:06.350
us use transpose notation for column vectors,
if a is invertible .
00:19:06.350 --> 00:19:23.309
Then we get x 1 x 2 x n is equal to A inverse
y 1 y 2 y n . In this case what is going to
00:19:23.309 --> 00:19:37.059
be the Jacobean .
00:19:37.059 --> 00:19:50.559
Let us write
A inverse is equal to B , is equal to b 1
00:19:50.559 --> 00:20:16.830
1, b 1 2, b 1 n , b n 1, b n 2 , b n n . Therefore,
x 1 x 2 up to xn is equal to b 1 1 up to b
00:20:16.830 --> 00:20:39.519
1 n b n 1 up to b n n times y 1 y 2 y n .
Therefore, the Jacobean is equal to del x
00:20:39.519 --> 00:21:00.159
1, del y 1, del x 1, del y 2, del xn, del
x 1, del y n, del xn, del y 1, del xn, del
00:21:00.159 --> 00:21:12.779
y 2, del xn, del y n .
Now, let us look at one of them. We have x
00:21:12.779 --> 00:21:33.290
1 is equal to b 1 1 y 1 plus b 1 2 y 2 plus
b 1 n y n . Therefore, del x 1 del y 1 is
00:21:33.290 --> 00:21:50.849
equal to b 1 1 del x 1 del y 2 is equal to
b 1 2 , del x 1 del y n is equal to b 1 n
00:21:50.849 --> 00:21:58.649
. Or in other words, the first row of the
Jacobean matrix is going to be the first row
00:21:58.649 --> 00:22:13.559
of b . In a similar way, we can see that this
Jacobean matrix j is nothing but the inverse
00:22:13.559 --> 00:22:24.369
matrix b.
Therefore the Jacobean of the transformation
00:22:24.369 --> 00:22:31.519
is equal to determinant of B which is equal
to determinant of A inverse, which is is equal
00:22:31.519 --> 00:22:39.870
to 1 upon determinant of A .
So, this is a very interesting observation
00:22:39.870 --> 00:23:10.601
. Now in particular suppose A is orthogonal
; that is, AA transpose is equal to I. A is
00:23:10.601 --> 00:23:19.139
an n cross n orthogonal matrix therefore,
aa transpose is equal to identity matrix,
00:23:19.139 --> 00:23:29.830
or A inverse is equal to A transpose . Therefore,
therefore, if we consider that transformation
00:23:29.830 --> 00:23:40.720
matrix to be orthogonal, we get this and we
also know that
00:23:40.720 --> 00:24:00.479
determinant of A is equal to 1 therefore,
joint pdf of y 1 y 2 y n is equal to joint
00:24:00.479 --> 00:24:17.659
pdf of x 1 x 2 xn multiplied by 1 this is
expressed in y i's right ?
00:24:17.659 --> 00:24:29.179
Let us now consider
00:24:29.179 --> 00:24:55.419
x 1 x 2 xn as independent samples from normal
mu sigma square . Therefore, f of x 1 x 2
00:24:55.419 --> 00:25:07.349
xn which is the joint pdf of x 1 x 2 xn is
equal to 1 over root over 2 pi sigma whole
00:25:07.349 --> 00:25:20.960
to the power n , e to the power minus sigma
x i minus mu whole square upon 2 sigma square
00:25:20.960 --> 00:25:35.419
, is equal to 1 over root over 2 pi to the
power n sigma to the power n e to the power
00:25:35.419 --> 00:25:49.009
minus half sigma x i minus mu upon sigma whole
square summation i is equal to 1 to n .
00:25:49.009 --> 00:26:21.289
Now, let us consider the following orthogonal
transformation .
00:26:21.289 --> 00:26:48.239
Y 1 is equal to 1 over root n 1 over root
n 1 over . So, y 1 is considered to be the
00:26:48.239 --> 00:26:55.529
dot product of 1 over root n, into x 1 plus
1 over root n into x 2 plus 1 over root n
00:26:55.529 --> 00:27:03.119
into x n .
Let us consider y 2 to be the second row is
00:27:03.119 --> 00:27:17.359
1 upon root 2 minus 1 upon root 2 0 0 . Therefore,
first thing you note that the sum of squares
00:27:17.359 --> 00:27:26.580
of these values is equal to 1. Sum of squares
of these values is equal to half plus half
00:27:26.580 --> 00:27:33.299
is equal to 1. Not only that if we look at
the dot product of this vector with this we
00:27:33.299 --> 00:27:39.669
get 0. Therefore, these 2 vectors are mutually
orthogonal .
00:27:39.669 --> 00:27:50.940
Similarly, we can have the third row is equal
to 1 over root 6, 1 over root 6 minus 2 over
00:27:50.940 --> 00:28:01.629
root 6 rest are 0. Again if you look at it
it is sum of square is equal to 1 by 6 plus
00:28:01.629 --> 00:28:11.309
1 by 6 plus 4 by 6 is equal to 1. And not
only that it is orthogonal to this and it
00:28:11.309 --> 00:28:28.489
is orthogonal to this .
In general, the ith row of the transformation
00:28:28.489 --> 00:28:49.039
matrix
is 1 over root I into I minus 1 into 1 1 1
00:28:49.039 --> 00:29:02.580
i minus 1 times minus I minus 1 and rest are
all 0's .
00:29:02.580 --> 00:29:13.919
Therefore, the sum square of the element is
1 upon i into i minus 1 plus 1 upon i into
00:29:13.919 --> 00:29:31.419
i minus 1 plus 1 upon i into I minus 1 . This
is i minus 1 times plus I minus 1 squared
00:29:31.419 --> 00:29:49.259
upon I into I minus 1 , is equal to 1 upon
i plus i minus 1 upon i is equal to 1. Thus
00:29:49.259 --> 00:29:59.880
the norm the length of each of the rows of
the matrix is 1 . You can easily verify that
00:29:59.880 --> 00:30:22.879
it is dot product with all the previous rules
going to be 0.
00:30:22.879 --> 00:31:12.989
Thus the matrix is an orthogonal matrix . Let
us use this matrix for our calculation . Let
00:31:12.989 --> 00:31:35.419
y 1 y 2 yn be a into x 1 minus mu upon sigma
up to xn minus mu upon sigma . So, this A
00:31:35.419 --> 00:31:40.110
is the particular A that I have just constructed
.
00:31:40.110 --> 00:32:02.539
Therefore, y 1 y 2 y n I can write it as a
1 1 upon sigma, a 1 2 upon sigma, a 1 n up
00:32:02.539 --> 00:32:22.889
a 1 n upon sigma , a n 1 upon sigma, a n 2
upon sigma, a n n upon sigma, multiplied by
00:32:22.889 --> 00:32:40.990
x 1 minus mu x 2 minus mu up to xn minus mu
. This a 1 is the original, this a ones are
00:32:40.990 --> 00:32:47.349
the matrix of the A transformation that we
have considered .
00:32:47.349 --> 00:33:04.730
Therefore, needs Jacobean is going to be 1
upon sigma to the power n , if I take the
00:33:04.730 --> 00:33:11.500
modulus if I take 1 upon sigma to be out from
each of the n columns the determinant is equal
00:33:11.500 --> 00:33:17.289
to 1. Therefore, that mod of the Jacobean,
if I consider this transformation that is
00:33:17.289 --> 00:33:39.700
going to be 1 upon sigma to the power n. Therefore,
if we consider
00:33:39.700 --> 00:33:56.649
the inverse of it , the Jacobean is going
to be sigma power n.
00:33:56.649 --> 00:34:18.240
Therefore joint pdf of y 1 y 2 y n is equal
to 1 over root over 2 pi sigma to the power
00:34:18.240 --> 00:34:31.440
n into e to the power minus half sigma x i
minus mu upon sigma whole square multiplied
00:34:31.440 --> 00:34:44.399
by sigma power n , is equal to 1 over root
over 2 pi whole to the power n e to the power
00:34:44.399 --> 00:34:59.319
minus half and it is sigma x i minus mu upon
sigma whole square , i is equal to 1 to n
00:34:59.319 --> 00:35:17.000
.
Now, y 1 y 2 yn is equal to A times x 1 minus
00:35:17.000 --> 00:35:24.599
mu upon sigma up to xn minus mu upon sigma
.
00:35:24.599 --> 00:35:40.730
Therefore y transpose y is equal to sigma
y i square 1 to n , is equal to x 1 minus
00:35:40.730 --> 00:35:56.140
mu upon sigma, xn minus mu upon sigma, into
A transpose A x 1 minus mu one sigma up to
00:35:56.140 --> 00:36:06.760
xn minus mu upon sigma , is equal to since
this is is equal to identity is equal to sigma
00:36:06.760 --> 00:36:28.380
x y x i minus mu upon sigma whole square . Therefore,
joint pdf of y 1 y 2 y n is equal to 1 over
00:36:28.380 --> 00:36:39.230
root over 2 pi whole to the power n e to the
power minus half sigma y i square i is equal
00:36:39.230 --> 00:37:04.349
to 1 to n . What does it tell us ?
It tells that y 1 1 y 2 y n are iid independent
00:37:04.349 --> 00:37:19.630
identically distributed random variables ; such
that each y i follow normal 0 1 . So, the
00:37:19.630 --> 00:37:27.010
advantage of the particular transformation
that I have made is that , it converts from
00:37:27.010 --> 00:37:39.180
x 1 x 2 xn each of which is normal mu sigma
square 2 y 1 y 2 y n which are independent,
00:37:39.180 --> 00:37:59.440
and each y is normal 0 1 .
Now, what is y 1 ? Y 1 is equal to x 1 minus
00:37:59.440 --> 00:38:16.950
mu upon root n sigma plus up to xn minus mu
upon root n sigma ,
00:38:16.950 --> 00:38:35.480
is equal to 1 over root n sigma x i minus
mu upon sigma , is equal to 1 upon sigma root
00:38:35.480 --> 00:38:53.609
n sigma x i is equal to n times x bar minus
n times mu , is equal to root n upon sigma
00:38:53.609 --> 00:39:06.019
into x bar minus mu, is equal to x bar minus
mu upon sigma root n .
00:39:06.019 --> 00:39:17.260
And since y 1 is normal 0 1 this is normal
0 1 as well. So, this also suggests that x
00:39:17.260 --> 00:39:28.880
bar is normal with mean is equal to mu, and
variance is equal to sigma over root n square
00:39:28.880 --> 00:39:40.750
is equal to sigma square by n . Although this
result I have proved before we can find it
00:39:40.750 --> 00:39:48.590
here also , but we get something extra . What
is that?
00:39:48.590 --> 00:40:00.750
Let us consider
sigma y i square 2 to n . This is is equal
00:40:00.750 --> 00:40:19.160
to sigma y i square 1 to n minus y 1 square
, is equal to sigma x i minus mu by sigma
00:40:19.160 --> 00:40:28.960
whole square i is equal to 1 to n. Because
we have seen some time back that sigma yi
00:40:28.960 --> 00:40:38.730
square is equal to sigma x i minus mu upon
sigma whole square, minus y 1 square which
00:40:38.730 --> 00:40:52.670
is is equal to n upon sigma square into x
bar minus mu whole square , is equal to 1
00:40:52.670 --> 00:41:09.980
upon sigma square into sigma x i minus mu
whole square minus n into x bar minus mu whole
00:41:09.980 --> 00:41:35.640
square , is equal to 1 by sigma square sigma
x i square minus 2 mu sigma x i plus n mu
00:41:35.640 --> 00:42:03.000
square, minus n x bar square plus 2 n mu x
bar plus minus n mu square . So, it is n times
00:42:03.000 --> 00:42:09.470
x bar square plus 2 mu x bar minus n times
mu square .
00:42:09.470 --> 00:42:32.630
So, this cancels is equal to 1 by sigma square
into sigma x i square minus 2 mu sigma x i
00:42:32.630 --> 00:42:52.470
minus n x bar square plus 2 n mu x bar . Now
sigma x i is equal to n times x bar. So, this
00:42:52.470 --> 00:43:04.779
also cancels out, is equal to 1 by sigma square
into sigma x i square minus n times x bar
00:43:04.779 --> 00:43:22.349
square .
On the other hand, , sigma x i minus x bar
00:43:22.349 --> 00:43:42.020
whole square is equal to sigma x i square
minus 2 x bar sigma x i plus n x bar square
00:43:42.020 --> 00:43:57.839
, is equal to sigma x i square minus 2 n x
bar square plus n x bar square is equal to
00:43:57.839 --> 00:44:12.710
sigma x i square minus n x bar square .
So, these 2 terms are same therefore we can
00:44:12.710 --> 00:44:24.859
write as sigma y to y i square from 2 to n
which we came out to be 1 by sigma square
00:44:24.859 --> 00:44:35.890
into sigma x i square minus n times x bar
square, is equal to 1 by sigma square into
00:44:35.890 --> 00:44:52.089
sigma x i minus x bar whole square .
Now, 1 by n sigma x i minus x bar whole square
00:44:52.089 --> 00:45:03.049
is equal to s square . Therefore, sigma x
i minus x bar whole square is equal to n times
00:45:03.049 --> 00:45:18.390
s square . Therefore, sigma y i square i is
equal to 2 to n is equal to n s square upon
00:45:18.390 --> 00:45:46.259
sigma square . This gives us 2 important consequences
.
00:45:46.259 --> 00:46:01.799
One ns square sigma square which is same as
n minus 1 into s square upon sigma square
00:46:01.799 --> 00:46:27.599
is sum of square
of n minus 1 independent normal 0 1 square
00:46:27.599 --> 00:46:39.550
; that is, n s square upon sigma square is
distributed as chi square with n minus 1 degrees
00:46:39.550 --> 00:46:49.181
of freedom. Because ns square upon sigma square
is sum of sigma yi square from 2 to n; that
00:46:49.181 --> 00:46:57.970
means, n minus 1 of independent normal 0 1
square. Therefore, ns square upon sigma square
00:46:57.970 --> 00:47:04.309
is distributed at chi square with n minus
1 degrees of freedom .
00:47:04.309 --> 00:47:20.020
And the second point is that n s square upon
sigma square is independent of y 1. Because
00:47:20.020 --> 00:47:29.430
we have found that y 1 y 2 yn are n independent
normal 0 1 variate of which n s square sigma
00:47:29.430 --> 00:47:37.310
square depends only on from y 2 to y n. Therefore,
it is independent of y 1 which is nothing
00:47:37.310 --> 00:48:04.890
but x bar minus mu upon sigma by root n .
Therefore, if x 1 x 2 xn are independent normal
00:48:04.890 --> 00:48:26.499
mu sigma square , then a x bar is normal with
mean mu variance sigma square by n , b sigma
00:48:26.499 --> 00:48:37.759
x i minus x bar whole square upon sigma square
is distributed as chi square with n minus
00:48:37.759 --> 00:48:54.910
1 degrees of freedom. And c distribution of
x bar and s square are independent .
00:48:54.910 --> 00:49:10.619
So, these are 3 important findings that we
get by making a particular orthogonal transformation
00:49:10.619 --> 00:49:22.410
of x 1 x 2 xn to y 1 y 2 y n. So, these are
some important results that we obtain when
00:49:22.410 --> 00:49:35.259
x 1 x 2 xn are independent samples from normal
mu sigma square . With that I stop for today.
00:49:35.259 --> 00:49:36.360
Thank you .