So, we have come to a situation where the problem is that, from the population we draw
a certain number of samples, say n samples, and a part of that we will be found to have
the value of 1, a part of that we will have the value of 0.
And we are trying to find out what proportion has the value of 1.
So, we have drawn n samples. Suppose the number of 1s in that n sample
is called X. So, X the number of 1’s in the sample of
n individuals. So, X is a fraction of n and we will now face
the problem of finding what is the mean of X, the standard deviation of X, and so on
and so forth. If I draw that n number of samples again and
again from that population, I will not get the same value of X every time.
I will get different values, and that those values of X will also have a distribution.
And we have just concluded that, if the number of samples becomes large, that distribution
will become more or less a normal distribution. And now we are we are facing the problem of
finding what is the mean of X, and what is the standard deviation of X.
So, our problem is is the mean of X and the standard deviation
of X. Now, X is the number of 1’s in the population
in in the in the sample of n individuals. Therefore, in finding out what X is, what
I am doing is that, I am picking up each individual from the sample and we are asking whether
it is 1 or 0? Again I am picking the second individual,
I am asking: 1 or 0? This is the same as what we are doing earlier.
That means, we are actually making a measurement of the Y’s.
Each measurement is Y, which can have two values 1 or 0.
We have seen that earlier. So, X, the measurement of X is actually n
measurements of Y. So, which is nothing but Y plus Y plus Y that
goes on n times. So, if that be so, we have already found out
what is the mean of Y. Then the mean of X is nothing but the mean
of Y, n times. So, then we can argue that, the the mean of
X can be simply expressed as the mean of Y plus the mean of Y plus the mean of Y ...
n times, and we know what the mean of Y is. We know that the mean of Y is p.
Therefore, n times of p is np. So, that should be the mean of X.
Now, what will be this variance? I would argue that, we would carry carry out
the variance exactly in the same way. So, the variance of Y variance of X, so, variance
of X should be again each measurement of X is nothing but n measurement of Y, and we
know the variance in the measurement of Y, and therefore, this is nothing but the n times
the variance of Y. Okay?
And the variance of Y we have already learned, n variance of Y is p into 1 minus p.
So, that should be the variance of X. So, we have a distribution whose the variance
is known. Therefore, the standard deviation standard
deviation would be square root of np 1 minus p.
Okay? Now, what are we actually trying to find out?
We are actually trying to find out the proportion the measured proportion.
The measured proportion
the measured proportion is that number that we have found: X is the number that we have
found, divided by the total number of samples. That is the measured proportion.
Let us call it measured therefore; it is not the proportion out there.
Let us call it with a hat. So, that we know that we are talking about
the measured proportion: the 1’s that we have measured using the number of, n number
of samples is ̂p̂ and that will be the that will be the measured proportion.
And if we do a large number of trials of taking n samples each time, we will get a distribution
of the measured proportions. For 1 1 sampling; that means, we have taken
n samples, we have got a measured proportion. If we do that again, taking another n samples,
we will get another value of this. A third time you will get another value of
this. Ultimately, the p̂ will have a distribution.
What will be the mean of that distribution? What will be the mean of the distribution
of the p̂’s? So, that will be let us call it mean of p̂.
That will be the mean of X mean of X divided by n.
Since the p is X by n, therefore, it will be this, and mean of X is something we know:
n times p. So, n times p by n.
Therefore, it will be p. So, the the... if I repeatedly make measurements,
each time taking n samples, I will get a distribution and that distribution will have a mean which
is the the proportion of the proportion ‘out there’ in the in the population.
So, that is a good thing. That we can obtain the the actual proportion
by making repeated measurements, every time taking n samples.
And what will be the standard deviation? The standard deviation, σ of p̂, that will
be the the σₓ divided by σₓ divided by there are n samples n.
Now, we know that σₓ was square root of np 1 minus p, and here is n.
Therefore, this comes to be square root of p 1 minus p by n.
So, the the conclusion is that if there is a proportion in the population out there and
I am trying to find it using a sampling process. I am drawing samples, n samples each time.
If I repeat the process again and again, then every time I will measure a different value
of the the measured proportion p̂, and that will have a distribution.
And that distribution will have a mean at the population mean p, and that mean that
p̂ will have a distribution whose standard deviation will be this, a well defined number.
Well you might ask: what is the guarantee that this distribution will be a normal distribution?
Earlier our argument for rooting on a normal distribution was the central limit theorem.
But here we cannot apply the central limit theorem, because it is a different problem
we are dealing with. So, how do we ensure that it will be a a normal
distribution? Now, it has been found that the question is
is legitimate. It has been found that, well it goes by a
by a rule of thumb. It is not not difficult to see that, if the
number of 1’s in the population is too small, it is like a dwindling dwindling population,
something close to extinction, if that is so, then the distribution will not be a a
normal distribution. So, there are situations where it will not
be a normal distribution. Therefore, we need to talk about under what
condition we can expect more or less a normal distribution.
And there is a rule of thumb for that. The rule of thumb is... this is rule of thumb.
It is not a a theorem. Just people have tried out and found that
it works. That is, if the probability p — and p is
a probability of getting a 1 — times times the number of samples
samples is greater than 10, and and if the the probability of the other one; that means,
1 minus p (1-p) times the number of samples, that is also greater than 10, then then the
distribution of the measured p̂ will be approximately
normal. If it is normal, then we can apply all that
we have learnt about the normal distribution. That is a major advantage.
But the caveat is that, that is not applicable to all cases.
For example, if if the if the... in the in the example that we have taken, in the example
that you have taken. p was 60 0.6, sorry I will do it right correctly p was 0.6, 1 minus
p was 0.4, and suppose we take 20 samples. n is equal to 20.
Then it will become p times 20, which is p into n is equal to... p into n will be 12
and 1 minus p into n will be 8. So, obviously, this is not bigger than ten
and therefore, in this collection the normal distribution will not apply.
So, if you keep on drawing samples of 20s and expect p̂ to be distributed in normal
distribution, that will not work. You have to take more than 20 samples.
Okay? Then only the normal distribution will work.
Before we end, let us quickly do an example. There are... consider the Mendel-type experiment
in which there are tall plants and and short plants, and you have collected a sample of
50. And you have counted the number and you have
found that 33 are are tall. I will write.
So, your your measured proportion is 33 by 50 is equal to 0.66.
The question is, can you state that the actual proportion out there lies in the range 0.64
to 0.68? Can you say...?
So, actual p lies in this range? This is the problem.
Okay? So, how do we attack this problem?
We know, the the mean will be... we have calculated the mean, sample mean, to be this, and we
can calculate the sample standard deviations: σ p̂ to be square root of p 1 minus p by
n. Now, p we do not know.
Therefore, we substitute by whatever we have measured, which is this.
So, it comes to be 0.66 times 1 minus 0.66 divided by the number of samples.
We have taken 50. Okay?
And that turns out to be 0.067. Okay?
Now, will this distribution be a normal distribution? Let’s quickly check: p is 0.66.
So, p into n is 0.66 times 50 is is 33, and 1 minus p times n is equal to 17.
So, both are bigger than 10. So, normal!
Okay? Distribution is normal.
That is good. So, if the distribution is normal, then we
can we can picture the distribution something like this.
And we know the mean will be at p, and the standard deviation we have just calculated.
Okay? The question was the probability of this.
So, probability of P of the the mean of p is — not p̂, p — is within 0.66 and 0.64
0.2 is within 0.2 of the mean: this is what we are trying to calculate.
So, p̂ 0.66 minus 0.2 and plus 0.2. So, our statement is: what is the probability
that this is true? So, this is again... sorry...
I will write this p as capital P because it is a probability.
Probability of again like the like the earlier problems, we will reverse the argument.
We will say p̂ is within 0.2 of the mean of p, the distance between this and that is
the same as distance between this and this. So, we have the same argument.
And then we can... we have to express that as a multiplier of the standard deviation.
Then we get the value of z. Okay?
So, the z value z is the multiplier 0.2, zero point...
sorry 0.2 or 0.02, 0.02 sorry 02 0.02 divided by the this is 0.067, and that turns out to
be 0.3. z turns out to 0.3.
So, we are essentially saying that p̂ is within 0.3 standard divisions of the the mean.
Now if you now consult the z table and find out what is the... suppose 0.3 is somewhere
here... 0.3, 0.3 would be 0.6 0.3 will be somewhere here may be.
So, if you find this area if you find this area then you will find that
the the area 0.6179. So, this area we will we will subtract 0.5
from it. So, 0.6179 minus 0.5, thereby we get only
this area times two. That turns out to be 0.2358.
So, that is the probability of having the actual the the proportion of the two populations
out there is to be to be in this range, to be in this range is only this much.
Which means that, I can state with only 23 percent confidence that the population will
be in this range. With that population would be in this range.
It is a very low confidence. That immediately tells us that we need to
change our strategy. We need to do something else in order to increase
the level of confidence for proportion measurements. I will stop here and continue with the next
class.