WEBVTT
Kind: captions
Language: en
00:00:00.740 --> 00:00:05.410
We are discussing the forward Kolmogorov equation
for a special case of Continuous-time Markov
00:00:05.410 --> 00:00:11.919
Chain that is the birth death process. For
a birth death process, the Q matrix is a tridiagonal
00:00:11.919 --> 00:00:17.199
matrix. Therefore, you will have the equations
from the forward Kolmogorov equation, you
00:00:17.199 --> 00:00:23.000
will have a only two terms in the right hand
side for the first equation and you will have
00:00:23.000 --> 00:00:29.960
only three terms, the diagonal element and
two off-diagonal elements.
00:00:29.960 --> 00:00:36.470
Therefore, the first equation one can discuss
first, the P dash of i,0 that is nothing,
00:00:36.470 --> 00:00:42.350
but the system is not moved from the state
zero, moving from the state zero that rate
00:00:42.350 --> 00:00:49.000
is lambda 0, therefore not moving minus lambda
0 times the probability and the system can
00:00:49.000 --> 00:00:58.480
come from the state one with the rate mu 1.
Therefore, mu 1 times P i,1 of t.
00:00:58.480 --> 00:01:04.570
For all other equations either the system
comes from the previous state with the rate
00:01:04.570 --> 00:01:15.299
lambda j minus 1 or it comes from the forward
one state with the rate mu j plus 1 or not
00:01:15.299 --> 00:01:20.469
moving anywhere. So these are all the all
possibilities therefore with these three possibilities
00:01:20.469 --> 00:01:27.899
you have a three terms in the right hand side
and that is the net rate for any strategy.
00:01:27.899 --> 00:01:32.840
So if you solve this equation with this initial
condition, Kronecker delta i comma j, you
00:01:32.840 --> 00:01:39.869
will have the solution of a P i,j.
00:01:39.869 --> 00:01:46.380
Here I am discussing the steady state distribution,
the way I have discussed the limiting distribution
00:01:46.380 --> 00:01:53.039
that is a limit t tends to infinity, probability
of i,j of t exist, then it is called the limiting
00:01:53.039 --> 00:01:59.509
distribution and the stationary distribution
is nothing but for the DTMC is pi is equal
00:01:59.509 --> 00:02:04.950
to pi P, summation of pi i is equal to 1.
For the CTMC pi Q is equal to zero and the
00:02:04.950 --> 00:02:10.870
summation of pi i is equal to 1. That is going
to be the steady state distribution, stationary
00:02:10.870 --> 00:02:12.000
distribution.
00:02:12.000 --> 00:02:17.750
Now I am discussing the steady state distribution,
that is nothing but when t tends to infinity
00:02:17.750 --> 00:02:24.650
the birth death process may reach steady state
or equilibrium condition. That means the state
00:02:24.650 --> 00:02:32.090
probability’s does not depend on time. That
is a meaning of a steady state distribution.
00:02:32.090 --> 00:02:38.170
As t tends to infinity, whenever we say the
birth death process reaches a steady state
00:02:38.170 --> 00:02:43.590
or at equilibrium, that state probability
does not depend on time.
00:02:43.590 --> 00:02:52.010
That means if a steady state solution exist,
since the state probability does not depend
00:02:52.010 --> 00:03:00.790
on time t, the derivative of the time dependent
state probability at time t, that derivative
00:03:00.790 --> 00:03:10.920
at t tends to infinity becomes zero. If the
steady state solution exists. Since the state
00:03:10.920 --> 00:03:15.820
probability’s does not depend on time t
as t tends to infinity, I can write as a pi
00:03:15.820 --> 00:03:20.000
i is a limit t tends to infinity of pi i of
t.
00:03:20.000 --> 00:03:25.370
So this is different from the way we discussed
earlier that conditional probability P i j
00:03:25.370 --> 00:03:32.530
of t. But using P i j of t, one can find out
what is pi j of t, pi i of t.
00:03:32.530 --> 00:03:40.000
That is nothing but the pi of t that I have
given in the first lecture for the CTMC. The
00:03:40.000 --> 00:03:45.400
pi i of t that is nothing but what is the
probability that the system will be in the
00:03:45.400 --> 00:03:54.160
state i at times. That is same as what is
the probability that the system will be in
00:03:54.160 --> 00:04:03.510
the state i given that it was in the state
some k at times zero multiplied by what is
00:04:03.510 --> 00:04:09.620
the probability that it was in the state k
at times.
00:04:09.620 --> 00:04:16.190
That is nothing but summation of k and this
is nothing but the transition probability
00:04:16.190 --> 00:04:28.389
and this is nothing but the initial probability
vector element. So using P ki of t or P ij
00:04:28.389 --> 00:04:32.610
of t that is a conditional probability, one
can get the unconditional probability. This
00:04:32.610 --> 00:04:36.030
is nothing but the distribution of X of t.
00:04:36.030 --> 00:04:43.610
So this is the probability mass function,
probability mass at state i. So now what I
00:04:43.610 --> 00:04:53.180
am defining, whenever the steady state distribution
exists, that means it is independent of time
00:04:53.180 --> 00:05:01.729
t. Therefore, as t tends to infinity the pi
i of t can be written as the pi i and whenever
00:05:01.729 --> 00:05:05.770
the steady state solution exists, I can use
limit t tends to infinity, the derivative
00:05:05.770 --> 00:05:08.480
of a pi i of t that is going to be zero.
00:05:08.480 --> 00:05:15.310
Therefore, I am going to use these two to
get the steady state probabilities for the
00:05:15.310 --> 00:05:24.210
birth death process. Since, as a t tends to
infinity, the derivative of pi i of t is equal
00:05:24.210 --> 00:05:29.319
to zero, therefore, all the left hand side
in the forward Kolmogorov equation that is
00:05:29.319 --> 00:05:36.430
going to be zero, the right hand side you
will have a as a t tends to infinity, the
00:05:36.430 --> 00:05:40.539
pi i of t, that can be written as the pi zero
and pi one.
00:05:40.539 --> 00:05:48.190
So the way we write the conditional probability
for P i j with the Kolmogorov forward equation,
00:05:48.190 --> 00:05:54.199
you can write the similar equation for the
unconditional probability pi i also. So now
00:05:54.199 --> 00:05:59.800
I am putting the left hand side zeros because
of this condition, limit t tends to infinity,
00:05:59.800 --> 00:06:05.469
the derivatives equal to zero and the right
hand side I am using as a t tends to infinity,
00:06:05.469 --> 00:06:07.639
this probability is nothing but the pi i.
00:06:07.639 --> 00:06:14.139
Therefore, it is going to be minus lambda0
times pi 0 plus mu 1 times pi and all other
00:06:14.139 --> 00:06:21.889
equation as a three terms. In this homogeneous
equation and you need a one normalising condition.
00:06:21.889 --> 00:06:28.560
So from this homogeneous equation, I can get
regressively pi i in terms of pi 0. So from
00:06:28.560 --> 00:06:33.499
the first equation, I can get a pi 1 in terms
of pi 0 and the second equation.
00:06:33.499 --> 00:06:41.009
I can get a pi 2 in terms of first pi 1 then
I can get a pi 1 in terms of pi 0. Therefore,
00:06:41.009 --> 00:06:47.449
regressively I can get pi i in terms of pi
0 for all i greater than or equal to 1.
00:06:47.449 --> 00:06:52.889
Now I can use a normalising condition, summation
of pi i is equal to 1, therefore, I will get
00:06:52.889 --> 00:07:00.890
a pi 0 is equal to 1 divided by summation
of this many terms in the product form. Since
00:07:00.890 --> 00:07:10.580
we need a steady state probabilities and all
the pi i are in terms of pi 0. As long as
00:07:10.580 --> 00:07:16.720
the denominator is converges, you will have
a pi 0 is greater than zero. So ones the pi
00:07:16.720 --> 00:07:21.400
0 is greater than zero, then you will get
all the pi i with the summation of pi i is
00:07:21.400 --> 00:07:23.059
equal to 1.
00:07:23.059 --> 00:07:30.870
So whenever these series converges, then I
will have a steady state distribution with
00:07:30.870 --> 00:07:37.150
the positive probability and a summation of
probability is going to be 1. So this is the
00:07:37.150 --> 00:07:42.629
condition for a steady state distribution
for a birth death process because we started
00:07:42.629 --> 00:07:48.219
with a birth death process forward Kolmogorov
equation using these two conditions we have
00:07:48.219 --> 00:07:54.610
simplified into this form and use a normalising
condition and get the pi 0.
00:07:54.610 --> 00:08:01.930
As long as the summation is or the series
is converges, then we will have the steady
00:08:01.930 --> 00:08:07.689
state. If the series diverges, that means
by substituting the values for the lambda
00:08:07.689 --> 00:08:15.189
i's and mu i’s and if the series denominator
series diverges, then the pi 0 is going to
00:08:15.189 --> 00:08:20.639
be zero in turn all the pi i’s are equal
to zero therefore the steady state distribution
00:08:20.639 --> 00:08:26.580
will not exist if the denominator series diverges.
00:08:26.580 --> 00:08:34.320
I am going to give a one simple result, for
a irreducible positive recurrent time homogeneous
00:08:34.320 --> 00:08:41.219
CTMC, we know that a limiting distribution
exist, a stationary distribution exist. Now
00:08:41.219 --> 00:08:48.110
I am including the steady state distribution
also exist, I have given for a steady state
00:08:48.110 --> 00:08:53.370
distribution for the birth rate process, not
for the CTMC but here I am giving the result
00:08:53.370 --> 00:08:58.990
for the CTMC. All the three distribution exist
and all are going to be same.
00:08:58.990 --> 00:09:05.690
Whenever the CTMC is a time homogeneous irreducible
positive recurrent, all these three distributions
00:09:05.690 --> 00:09:11.760
are same and one can evaluate, one can solve
this two equation by pi Q is equal to zero
00:09:11.760 --> 00:09:18.910
and with the summation of pi is equal to 1,
you can get the limiting distribution, stationary
00:09:18.910 --> 00:09:24.370
distribution or steady state or equilibrium
distribution.
00:09:24.370 --> 00:09:29.210
As a special case of birth death process,
I am going to discuss these two process in
00:09:29.210 --> 00:09:32.040
this lecture.
00:09:32.040 --> 00:09:38.600
Whenever, we say the birth death process is
a pure birth process, that means all the death
00:09:38.600 --> 00:09:44.170
rates are going to be zero, we started with
a birth death process with the only lambda
00:09:44.170 --> 00:09:48.690
i’s are greater than zero and the mu i’s
are going to be zero, then it is going to
00:09:48.690 --> 00:09:53.620
be call it as a pure birth process. There
is a one special case of pure birth process
00:09:53.620 --> 00:09:58.760
with the lambda i’s are going to be constant,
that is lambda, that is a Poisson process.
00:09:58.760 --> 00:10:04.259
I am going to discuss in the next lecture
and in these pure birth process, these lambda
00:10:04.259 --> 00:10:13.410
i’s are the function of i. Here all the
states are transient states.
00:10:13.410 --> 00:10:20.050
Here I am discussing the pure death process.
A birth death process is said to be a pure
00:10:20.050 --> 00:10:27.959
death process if the birth rates are zero
and the death rates are non zero. In particular,
00:10:27.959 --> 00:10:34.670
we shall obtain the time dependent probabilities
of a pure death process in which the death
00:10:34.670 --> 00:10:39.920
rates mu i’s are equal to i times mu.
00:10:39.920 --> 00:10:49.069
As I given the example, as a fourth example
in the birth death process, this state zero
00:10:49.069 --> 00:10:55.940
is a absorbing barrier. Therefore, the state
zero is absorbing state and all other states
00:10:55.940 --> 00:11:01.769
are going to be transient state. And here
the limiting distribution exists and one can
00:11:01.769 --> 00:11:06.589
also find time dependent probabilities for
this model.
00:11:06.589 --> 00:11:12.089
Suppose you start with the assumption, the
system at time zero, in the system is in the
00:11:12.089 --> 00:11:19.940
state n, at times zero the system in the state
n at times zero. With that assumption, I can
00:11:19.940 --> 00:11:26.519
frame the equation that is the pi n dash of
t is equal to minus n times mu of pi n of
00:11:26.519 --> 00:11:27.519
t.
00:11:27.519 --> 00:11:33.670
That means the rate in which the system is
in the state n that is nothing but not moving
00:11:33.670 --> 00:11:40.180
to the state n minus 1 with the rate n minus
n times mu. Therefore, the equation for the
00:11:40.180 --> 00:11:47.800
state n that is a pi n dash of t that is equal
to not moving from the state n therefore minus
00:11:47.800 --> 00:11:54.800
that outgoing rate that is n times mu being
the state is n therefore pi n of t. I can
00:11:54.800 --> 00:12:01.009
use the initial condition pi n of zero is
equal to 1, so I will get pi n of t.
00:12:01.009 --> 00:12:06.839
For the second equation, I have to go for
what is the equation for the state n minus
00:12:06.839 --> 00:12:15.230
1. So the pi n minus 1 dash t, that is nothing
but either the system come from the state
00:12:15.230 --> 00:12:21.740
n or not moving from the state n minus 1.
Therefore, system coming from the state n
00:12:21.740 --> 00:12:32.140
that is a n mu times the system being the
state n minus n minus 1 times mu pi n minus
00:12:32.140 --> 00:12:39.850
1 of t. So we will have a two terms in the
right hand side coming from the one forward
00:12:39.850 --> 00:12:43.060
state or not moving from the same state.
00:12:43.060 --> 00:12:50.980
So you will have a two terms for j is equal
to 1 to n minus 1. For the last state, that
00:12:50.980 --> 00:12:58.240
is the state zero, the system comes from the
state 1. Since the state zero is absorbing
00:12:58.240 --> 00:13:05.330
states, there is no second term. So it is
going to be mu times pi n of t. So you know
00:13:05.330 --> 00:13:13.920
pi n of t, use the pi n of t in the equation
for n minus 1 and get the pi n minus 1, like
00:13:13.920 --> 00:13:24.889
that you find out till pi 1. Use the pi 1
to get the pi 0 of t. Use the recursive way.
00:13:24.889 --> 00:13:30.829
So using the recursive way, you will get the
pi j of t is equal to n C j combination n
00:13:30.829 --> 00:13:38.100
C j and e power minus mu times t power j,
this is survival probability of system being
00:13:38.100 --> 00:13:46.680
in the state and 1 minus e power minus mu
of t n minus j. Suppose the system being in
00:13:46.680 --> 00:13:54.279
the state j, that means from the state n this
many combinations would have come and the
00:13:54.279 --> 00:13:59.510
survival probability is e power minus mu times
t and that power.
00:13:59.510 --> 00:14:06.009
So this is nothing but the probability p power
j and one minus p power n minus j. Therefore,
00:14:06.009 --> 00:14:12.199
this pi j follows the binominal distribution
with the survival probability e power minus
00:14:12.199 --> 00:14:16.350
mu t being in the state j.
00:14:16.350 --> 00:14:22.730
So for the pure death process, I have explained
the time dependent probabilities of being
00:14:22.730 --> 00:14:29.959
in the state j, that is a unconditioned probability.
So with this the summary of this lecture is,
00:14:29.959 --> 00:14:36.360
I have discussed the limiting, stationary
and a steady state distribution, I have introduced
00:14:36.360 --> 00:14:42.019
a birth death process. Some important results
also discussed.
00:14:42.019 --> 00:14:47.129
And at the end, I have discussed the pure
birth and pure death process also. In the
00:14:47.129 --> 00:14:54.670
next lecture, I am going to explain the important
pure birth process that is the Poisson process.
00:14:54.670 --> 00:15:09.199
And these are all the reference books. Thanks.