WEBVTT
Kind: captions
Language: en
00:00:00.539 --> 00:00:09.450
How to study the Stationary Distribution for
a reducible Markov chain along with the assumptions
00:00:09.450 --> 00:00:16.010
one is aperiodic and the finite state.
00:00:16.010 --> 00:00:23.310
Here I am making one more, here I am giving
the stationary distribution.
00:00:23.310 --> 00:00:30.770
So I am giving the result for a reducible
finite Markov chain, Markov chain is as in
00:00:30.770 --> 00:00:36.130
the finite state space and it is reducible
one.
00:00:36.130 --> 00:00:42.320
With the closed communicating class has aperiodic
states.
00:00:42.320 --> 00:00:50.190
There is a mistake the closed communicating
class of state has aperiodic states, aperiodic
00:00:50.190 --> 00:00:56.930
the closed communicating class of states as
aperiodic then the stationary distribution
00:00:56.930 --> 00:01:03.350
exists that is going to be unique also and
that is given by the vector V which consist
00:01:03.350 --> 00:01:10.750
of two sub vectors V1, 0 vector that you can
find out.
00:01:10.750 --> 00:01:16.920
And this is nothing but the Ergodic Theorem
at the reducible Markov chain with the assumption
00:01:16.920 --> 00:01:23.920
finite state space and the states of closed
communicating class has aperiodic states.
00:01:23.920 --> 00:01:31.030
In that case you get the unique stationary
distribution and that unique stationary distribution
00:01:31.030 --> 00:01:37.190
has a two sub one that vectors are V1 and
vectors of 0.
00:01:37.190 --> 00:01:43.990
Before you can get the stationary distribution
you can find out what is the n-step transition
00:01:43.990 --> 00:01:47.659
probability for the same reducible Markov
chain model.
00:01:47.659 --> 00:01:55.810
So the P1 is going to be, you have a sub matrix
Stochastic sub matrix P1 therefore that is
00:01:55.810 --> 00:02:03.480
going to be P1 power n whereas for every n
this is going to be a function of n.
00:02:03.480 --> 00:02:12.260
R is the sub matrix which is the one step
going from the transient state to the closed
00:02:12.260 --> 00:02:14.299
communicating class.
00:02:14.299 --> 00:02:23.650
Now the R power n is nothing but a function
of n that element that sub matrix is corresponding
00:02:23.650 --> 00:02:26.849
to the transient state to the communicating
class.
00:02:26.849 --> 00:02:31.680
Whereas the transient to transient that is
going to be a power n.
00:02:31.680 --> 00:02:40.329
It is a Q matrix Q matrix is the sub matrix
for one step T to T whereas the Q power n
00:02:40.329 --> 00:02:45.719
is the element corresponding to the n step
transition probability matrix.
00:02:45.719 --> 00:02:55.560
So as n tends to infinite the Stochastic sub
matrix that power n that will the vector of
00:02:55.560 --> 00:03:06.749
V1, V1 is the sub few elements that is corresponding
to the stationary state probability for the
00:03:06.749 --> 00:03:09.959
state corresponding to the closed communicating
class of states.
00:03:09.959 --> 00:03:19.739
So e is the vector of the entries one, one,
one and so on multiplied by the V1.
00:03:19.739 --> 00:03:27.989
And the transient to transient n-Step transition
probability as n tends to infinity this will
00:03:27.989 --> 00:03:30.040
tends to 0 this is obvious.
00:03:30.040 --> 00:03:36.650
Because the states are transient states for
a finite n you have a probability Q power
00:03:36.650 --> 00:03:43.519
n whereas as n tends to infinite the system
will not be in the transient state.
00:03:43.519 --> 00:03:48.839
Therefore, the long run proportion of the
time the system being in the transient states
00:03:48.839 --> 00:03:51.290
that is 0 as n tends to infinite.
00:03:51.290 --> 00:03:58.150
Therefore, Q power n tends to 0 or this will
tens the to the stationary state probabilities.
00:03:58.150 --> 00:04:06.319
Therefore, this stationary distribution vector
V consist of few elements of zeros that is
00:04:06.319 --> 00:04:14.000
corresponding to a transient states, transient
state probabilities in a long run.
00:04:14.000 --> 00:04:21.760
And V1 is the steady state probabilities in
a longer run it is not steady state its stationary
00:04:21.760 --> 00:04:30.050
distribution probability in a longer run for
the closed communicating class of states.
00:04:30.050 --> 00:04:43.150
So this one can solve by using equation pi
P is equal to pi you can get this pi’s that
00:04:43.150 --> 00:04:50.210
pi’s is in the notation and here it is Vi,
V1.
00:04:50.210 --> 00:04:58.140
So now I am making a further assumption the
states are going to be the positive recurrent.
00:04:58.140 --> 00:05:06.180
So already I made a aperiodic states now I
am making one more assumption it is a positive
00:05:06.180 --> 00:05:07.180
recurrent.
00:05:07.180 --> 00:05:13.161
Once it is a positive recurrent then the limiting
probability is limit n tends to infinite that
00:05:13.161 --> 00:05:20.980
probability is going to be Vj for the positive
recurrent states and for all the transient
00:05:20.980 --> 00:05:24.570
states the probabilities are going to be zero.
00:05:24.570 --> 00:05:29.660
And since we have a reducible Markov chain
with one closed communicating class.
00:05:29.660 --> 00:05:37.450
And all other states are transient states
this stationary distribution stationary state
00:05:37.450 --> 00:05:42.450
probabilities these probabilities are independent
of the initial state i.
00:05:42.450 --> 00:05:49.530
That means either the system you can start
a time zero in the one of the states in closed
00:05:49.530 --> 00:05:52.990
communicating class or transient states.
00:05:52.990 --> 00:05:59.520
In a longer run, ultimately the system will
be in one of the states in the closed communicating
00:05:59.520 --> 00:06:06.900
class whether it started initially from the
closed communicating class or transient state.
00:06:06.900 --> 00:06:16.170
Therefore, this stationary distribution is
independent of initial state i and for transient
00:06:16.170 --> 00:06:21.990
state you can conclude immediately these probabilities
are zeros and for a positive recurrent states
00:06:21.990 --> 00:06:27.920
you can make it Vj and you can compute this
Vj.
00:06:27.920 --> 00:06:38.430
Now I am going to give one simple example
in which we have a infinite state.
00:06:38.430 --> 00:06:46.090
This is going to be a reducible Markov chain
because the states till five not till five
00:06:46.090 --> 00:06:52.810
including four and two the system come to
the state three there is no arc from three
00:06:52.810 --> 00:07:03.280
to four or three to two therefore the states
2,4,5,6 and so on all those states are transient
00:07:03.280 --> 00:07:09.500
states whereas the state one and three are
going to form a one closed communicating class.
00:07:09.500 --> 00:07:14.870
Therefore, this is the reducible Markov chain
with the one closed communicating class one
00:07:14.870 --> 00:07:21.370
and three and all other states are going to
be transient states therefore as n tends to
00:07:21.370 --> 00:07:30.510
infinite this probability is are going to
be zero for these states 2,4,5 and so on and
00:07:30.510 --> 00:07:33.460
these probabilities are independent of the
initial state i.
00:07:33.460 --> 00:07:39.090
So wherever the i whether i is belonging to
the one of the event one of the states in
00:07:39.090 --> 00:07:44.260
the closed communicating class of states or
the transient state immaterial of that this
00:07:44.260 --> 00:07:53.080
is stationary distribution are zeros for the
transient state for the closed communicating
00:07:53.080 --> 00:08:01.560
class of states you can find out this probability
by separately making the Markov chain.
00:08:01.560 --> 00:08:13.260
The states one and three you can make it separately
and there is a arc from one to three with
00:08:13.260 --> 00:08:20.639
the probability half there is self-loop with
the probability one by two and there is a
00:08:20.639 --> 00:08:27.590
self-loop in the state three with the probability
one by three and the arc from three to one
00:08:27.590 --> 00:08:30.180
is two by three.
00:08:30.180 --> 00:08:35.740
So what do you want to find out this stationary
distribution for this two states.
00:08:35.740 --> 00:08:42.219
Therefore, you make as stochastic sub matrix
with the states one and three.
00:08:42.219 --> 00:08:48.649
That is one by two, one by three, two by three
and one by three this is also stochastic matrix
00:08:48.649 --> 00:08:50.379
you can verify.
00:08:50.379 --> 00:08:55.959
Now if you want to find out the stationary
distribution for these two states you solve
00:08:55.959 --> 00:09:00.470
by pi P1 is equal to pi.
00:09:00.470 --> 00:09:14.190
That means pi 1, pi 3 times P1 that is one
by two, one by three oh sorry, one by two,
00:09:14.190 --> 00:09:23.529
this is one by two, one by two, one by two
and this is two by three; one by three that
00:09:23.529 --> 00:09:27.129
is equal to pi 1, pi 3.
00:09:27.129 --> 00:09:39.470
You take the first equation that is pi 1,
half times pi 1one plus two third pi 3 that
00:09:39.470 --> 00:09:43.180
is equal to pi one.
00:09:43.180 --> 00:09:51.230
So from here you will get pi three is equal
to three by four pi 1.
00:09:51.230 --> 00:09:57.580
Now we use a pi 1 plus pi 3 is equal to one.
00:09:57.580 --> 00:10:03.420
So using this we will get pi 1 is equal to
four by seven.
00:10:03.420 --> 00:10:12.059
Once you know the pi 1 the pi 3 is going to
be three by seven.
00:10:12.059 --> 00:10:18.300
So you do not want to find out stationary
distribution for the whole model instead of
00:10:18.300 --> 00:10:22.339
that you can find out what is the closed communicating
class.
00:10:22.339 --> 00:10:28.389
And you can solve only a closed communicating
class that sub matrix pi P1 is equal to pi
00:10:28.389 --> 00:10:35.839
and you will get pi 1 and pi 3 and that is
going to be in a longer run that is equal
00:10:35.839 --> 00:10:41.480
to four by seven and three by seven and all
other states are going to be zero.