WEBVTT
Kind: captions
Language: en
00:00:00.299 --> 00:00:07.590
Now I am going to discuss the simple situation
in which how we can get the limiting state
00:00:07.590 --> 00:00:17.499
probabilities, this is a simple model in which
we have a only two states and this two states
00:00:17.499 --> 00:00:27.569
model is the very good example in the sense
this can be interpreted as the many situation.
00:00:27.569 --> 00:00:35.190
For example, you can think of a weather problem
in which 0 is for a rainy day and 1 is for
00:00:35.190 --> 00:00:36.190
the sunny day.
00:00:36.190 --> 00:00:40.809
And what is the probability that the next
day is going to be a sunny day from the rainy
00:00:40.809 --> 00:00:46.710
day that probability is ‘a’ and from rainy
day to sunny day it is going to be the probability
00:00:46.710 --> 00:00:52.600
‘b’ and the next day is going to be the
same thing whether it is a rainy day or sunny
00:00:52.600 --> 00:00:59.719
day according to the probability is 1 - a
and 1 - b. And you can assume that both the
00:00:59.719 --> 00:01:04.190
probabilities a and b lies between open interval
0 to1.
00:01:04.190 --> 00:01:11.370
In this case, this is a very simple two state
model like this we can give many more applications
00:01:11.370 --> 00:01:15.910
can be interpreted with the two state model
with the transition probability, this is a
00:01:15.910 --> 00:01:20.440
one step transition probability with a P matrix.
00:01:20.440 --> 00:01:37.400
That is the P matrix is the state 0 and 1
- 0 and 1, so 0 to 0 1 - a, 0 to 1 that probability
00:01:37.400 --> 00:01:46.040
a and 1 to 0 that property is b and 1 to 1
that is probability 1 - b, so this is a one
00:01:46.040 --> 00:01:51.310
step transition probability matrix and from
this model you can see that it is since a
00:01:51.310 --> 00:02:01.200
and b is open interval 0 to 1, this is going
to be a irreducible Markov chain and with
00:02:01.200 --> 00:02:06.410
a finite state space therefore using the result
we can conclude all the states are going to
00:02:06.410 --> 00:02:08.340
be a positive recurrent.
00:02:08.340 --> 00:02:13.810
That can be verified from the classification
of the states also, you can verify that first
00:02:13.810 --> 00:02:21.070
one is recurrent state that means you can
find out the probability of F 00 that is going
00:02:21.070 --> 00:02:29.090
to be 1 and similarly, you can find out F
11 that is also going to be 1, so you can
00:02:29.090 --> 00:02:33.930
conclude both the states are going to be a
positive recurrent and you can find out Mu
00:02:33.930 --> 00:02:41.120
0 0, that is going to be a finite quantity
as well as a Mu 1 1, that is also going to
00:02:41.120 --> 00:02:42.530
be a finite quantity.
00:02:42.530 --> 00:02:47.180
Therefore, you can conclude it is going to
be a positive recurrent, now our interest
00:02:47.180 --> 00:02:52.319
is what is the limiting distribution that
means you find out what is a limiting distribution
00:02:52.319 --> 00:02:58.440
matrix that is nothing but a limit n tends
to infinity P power (n), where P power (n)
00:02:58.440 --> 00:03:05.500
is nothing but the n step transition probability
matrix that is same as a one step transition
00:03:05.500 --> 00:03:11.750
probability matrix the power n, that means
you have to find out what is a P power n for
00:03:11.750 --> 00:03:12.750
any n.
00:03:12.750 --> 00:03:20.450
Then, you have to find out what is the P power
n matrix as n tends to infinity, so you can
00:03:20.450 --> 00:03:28.560
use a either eigenvalue and eigenvector method
or you can use the by induction method that
00:03:28.560 --> 00:03:35.240
means you find out P power 2 then P power
3 and so on, then you find out what is the
00:03:35.240 --> 00:03:41.569
P power n by Mathematical Induction or you
find out the eigenvalues or eigenvectors then
00:03:41.569 --> 00:03:46.099
you find out the P power n.
00:03:46.099 --> 00:03:53.820
So here I am directly giving the P power n
values matrix so this consists of 4 elements
00:03:53.820 --> 00:04:03.500
with a function of a, b and n, this will exist
provided the absolute of 1 - a - b is less
00:04:03.500 --> 00:04:10.470
than 1, otherwise this won't P power n won't
exist, now we are going for as n tends to
00:04:10.470 --> 00:04:14.230
infinity what is the matrix?
00:04:14.230 --> 00:04:20.239
That is limit n tends to infinity the P power
n is that matrix is going to be again it is
00:04:20.239 --> 00:04:25.680
going to be a stochastic matrix, because the
row sum is going to be 1 and all the elements
00:04:25.680 --> 00:04:32.960
are greater than or equal to 0, therefore
if the limiting probability matrix exist,
00:04:32.960 --> 00:04:37.419
then it is going to be unique at the limit
exists means it is unique and the row sums
00:04:37.419 --> 00:04:43.199
are - row values are all the rows are going
to be identical that you can visualize.
00:04:43.199 --> 00:04:52.180
So that vector is going to be pi, that is
pi 0 and pi 1, so the pi 0 is nothing but
00:04:52.180 --> 00:04:59.460
b divided by a + b and pi 1 is nothing but
a divided by a + b, these are all the limiting
00:04:59.460 --> 00:05:10.610
state probability that means in a longer run
the system will be in the state 0 or in the
00:05:10.610 --> 00:05:16.910
state 1 and the system will be in the state
0 in a longer run with the probability b divided
00:05:16.910 --> 00:05:22.789
by a + b, in a longer run the system will
be in the state 1 with the probability a divided
00:05:22.789 --> 00:05:24.669
by a + b.
00:05:24.669 --> 00:05:31.980
Note that these probabilities are independent
of a initial state i that means whether you
00:05:31.980 --> 00:05:39.900
start at time 0 in the state 0 or 1 does not
matter in a longer run the system is going
00:05:39.900 --> 00:05:47.719
to be in the state 0 or 1 with these probabilities,
so this is the situation for a time homogeneous
00:05:47.719 --> 00:05:54.090
discrete time Markov chain with the finite
state space and irreducible Markov chain.
00:05:54.090 --> 00:05:56.119
Therefore, all the states are positive recurrent.
00:05:56.119 --> 00:06:02.500
And we are getting the limiting state probabilities
which are all going to be independent of initial
00:06:02.500 --> 00:06:09.449
state so this information is going to be useful
later based on this I am going to distinguish
00:06:09.449 --> 00:06:14.930
three different probabilities distribution,
the one is a limiting distribution the next
00:06:14.930 --> 00:06:21.590
one is the stationary distribution the third
one is the steady-state or equilibrium distribution.
00:06:21.590 --> 00:06:29.339
In general, all these three results are all
these three distributions are different that
00:06:29.339 --> 00:06:35.930
is the limiting distributions, stationary
distributions and steady state or equilibrium
00:06:35.930 --> 00:06:40.860
distributions all three are different in general,
but there are in some situations that means
00:06:40.860 --> 00:06:48.129
for a special case of discrete time Markov
chain all these three results are going to
00:06:48.129 --> 00:07:00.110
be same so for that this example is going
to be important one. Now I am going to discuss
00:07:00.110 --> 00:07:01.779
the ergodicity.
00:07:01.779 --> 00:07:10.749
This is a very important concept in the any
dynamical system, but here we are discussing
00:07:10.749 --> 00:07:17.180
the Markov process or the - or we are going
to discuss the time homogeneous discrete time
00:07:17.180 --> 00:07:29.659
Markov Chain but the ergodicity is important
concept for any dynamical system, so I can
00:07:29.659 --> 00:07:35.740
give the easy definition that is it is necessary
and sufficient condition for existence of
00:07:35.740 --> 00:07:40.460
Vj’s, that is nothing but a some probability
state probabilities.
00:07:40.460 --> 00:07:47.189
If that is satisfying Vj's are going to the
summation Vi's * P ij and the Vi's are going
00:07:47.189 --> 00:07:53.689
to be summation is going to be 1 for j, in
case of irreducible aperiodic Markov chain
00:07:53.689 --> 00:08:03.569
then we are going to say the system is a ergodic
system that means whenever the system is irreducible
00:08:03.569 --> 00:08:11.400
and aperiodic Markov chain and then that system
is going to be call it as an ergodic Markov
00:08:11.400 --> 00:08:18.569
chain or this process is called the ergodicity.
00:08:18.569 --> 00:08:24.449
That means a if you have a irreducible and
aperiodic Markov chain the ergodicity property
00:08:24.449 --> 00:08:31.509
is satisfied, what is the use of ergodicity
property in the Markov chain? Since it is
00:08:31.509 --> 00:08:37.810
a irreducible and aperiodic this limiting
distributions this probabilities are going
00:08:37.810 --> 00:08:46.050
to be independent of initial state therefore
this is used in the discrete event simulation.
00:08:46.050 --> 00:08:50.709
That means if you want to find out the what
is the proportion of the time the system being
00:08:50.709 --> 00:08:57.279
in some state in a longer than that you can
compute by finding the that is nothing but
00:08:57.279 --> 00:09:02.230
the limiting probabilities and this limiting
properties same as this probabilities Vj's
00:09:02.230 --> 00:09:08.680
can be computed in this way using the one
step transition probability matrix and that
00:09:08.680 --> 00:09:12.380
probability is going to be always the independent
of an initial distribution.
00:09:12.380 --> 00:09:18.699
That means whatever the you are going to provide
the discrete event simulation that does not
00:09:18.699 --> 00:09:24.130
matter and you are interested only in the
longer run what is the proportion of the time
00:09:24.130 --> 00:09:30.740
the system being in some state, so that can
be easily computed for a ergodic system that
00:09:30.740 --> 00:09:36.670
means before you use the ergodic property
in the any dynamical system, you have to make
00:09:36.670 --> 00:09:44.600
sure that - that system is a irreducible aperiodic,
then you can use the ergodicity concept.