Now I am going to discuss the simple situation in which how we can get the limiting state
probabilities, this is a simple model in which we have a only two states and this two states
model is the very good example in the sense this can be interpreted as the many situation.
For example, you can think of a weather problem in which 0 is for a rainy day and 1 is for
the sunny day.
And what is the probability that the next day is going to be a sunny day from the rainy
day that probability is ‘a’ and from rainy day to sunny day it is going to be the probability
‘b’ and the next day is going to be the same thing whether it is a rainy day or sunny
day according to the probability is 1 - a and 1 - b. And you can assume that both the
probabilities a and b lies between open interval 0 to1.
In this case, this is a very simple two state model like this we can give many more applications
can be interpreted with the two state model with the transition probability, this is a
one step transition probability with a P matrix.
That is the P matrix is the state 0 and 1 - 0 and 1, so 0 to 0 1 - a, 0 to 1 that probability
a and 1 to 0 that property is b and 1 to 1 that is probability 1 - b, so this is a one
step transition probability matrix and from this model you can see that it is since a
and b is open interval 0 to 1, this is going to be a irreducible Markov chain and with
a finite state space therefore using the result we can conclude all the states are going to
be a positive recurrent.
That can be verified from the classification of the states also, you can verify that first
one is recurrent state that means you can find out the probability of F 00 that is going
to be 1 and similarly, you can find out F 11 that is also going to be 1, so you can
conclude both the states are going to be a positive recurrent and you can find out Mu
0 0, that is going to be a finite quantity as well as a Mu 1 1, that is also going to
be a finite quantity.
Therefore, you can conclude it is going to be a positive recurrent, now our interest
is what is the limiting distribution that means you find out what is a limiting distribution
matrix that is nothing but a limit n tends to infinity P power (n), where P power (n)
is nothing but the n step transition probability matrix that is same as a one step transition
probability matrix the power n, that means you have to find out what is a P power n for
any n.
Then, you have to find out what is the P power n matrix as n tends to infinity, so you can
use a either eigenvalue and eigenvector method or you can use the by induction method that
means you find out P power 2 then P power 3 and so on, then you find out what is the
P power n by Mathematical Induction or you find out the eigenvalues or eigenvectors then
you find out the P power n.
So here I am directly giving the P power n values matrix so this consists of 4 elements
with a function of a, b and n, this will exist provided the absolute of 1 - a - b is less
than 1, otherwise this won't P power n won't exist, now we are going for as n tends to
infinity what is the matrix?
That is limit n tends to infinity the P power n is that matrix is going to be again it is
going to be a stochastic matrix, because the row sum is going to be 1 and all the elements
are greater than or equal to 0, therefore if the limiting probability matrix exist,
then it is going to be unique at the limit exists means it is unique and the row sums
are - row values are all the rows are going to be identical that you can visualize.
So that vector is going to be pi, that is pi 0 and pi 1, so the pi 0 is nothing but
b divided by a + b and pi 1 is nothing but a divided by a + b, these are all the limiting
state probability that means in a longer run the system will be in the state 0 or in the
state 1 and the system will be in the state 0 in a longer run with the probability b divided
by a + b, in a longer run the system will be in the state 1 with the probability a divided
by a + b.
Note that these probabilities are independent of a initial state i that means whether you
start at time 0 in the state 0 or 1 does not matter in a longer run the system is going
to be in the state 0 or 1 with these probabilities, so this is the situation for a time homogeneous
discrete time Markov chain with the finite state space and irreducible Markov chain.
Therefore, all the states are positive recurrent.
And we are getting the limiting state probabilities which are all going to be independent of initial
state so this information is going to be useful later based on this I am going to distinguish
three different probabilities distribution, the one is a limiting distribution the next
one is the stationary distribution the third one is the steady-state or equilibrium distribution.
In general, all these three results are all these three distributions are different that
is the limiting distributions, stationary distributions and steady state or equilibrium
distributions all three are different in general, but there are in some situations that means
for a special case of discrete time Markov chain all these three results are going to
be same so for that this example is going to be important one. Now I am going to discuss
the ergodicity.
This is a very important concept in the any dynamical system, but here we are discussing
the Markov process or the - or we are going to discuss the time homogeneous discrete time
Markov Chain but the ergodicity is important concept for any dynamical system, so I can
give the easy definition that is it is necessary and sufficient condition for existence of
Vj’s, that is nothing but a some probability state probabilities.
If that is satisfying Vj's are going to the summation Vi's * P ij and the Vi's are going
to be summation is going to be 1 for j, in case of irreducible aperiodic Markov chain
then we are going to say the system is a ergodic system that means whenever the system is irreducible
and aperiodic Markov chain and then that system is going to be call it as an ergodic Markov
chain or this process is called the ergodicity.
That means a if you have a irreducible and aperiodic Markov chain the ergodicity property
is satisfied, what is the use of ergodicity property in the Markov chain? Since it is
a irreducible and aperiodic this limiting distributions this probabilities are going
to be independent of initial state therefore this is used in the discrete event simulation.
That means if you want to find out the what is the proportion of the time the system being
in some state in a longer than that you can compute by finding the that is nothing but
the limiting probabilities and this limiting properties same as this probabilities Vj's
can be computed in this way using the one step transition probability matrix and that
probability is going to be always the independent of an initial distribution.
That means whatever the you are going to provide the discrete event simulation that does not
matter and you are interested only in the longer run what is the proportion of the time
the system being in some state, so that can be easily computed for a ergodic system that
means before you use the ergodic property in the any dynamical system, you have to make
sure that - that system is a irreducible aperiodic, then you can use the ergodicity concept.