WEBVTT
Kind: captions
Language: en
00:00:14.410 --> 00:00:20.550
So, I am going to talk about discrete stochastic
processes, and without, you know,
00:00:20.550 --> 00:00:28.040
spending time on first trying to define stochastic
processes and on that discrete stochastic
00:00:28.040 --> 00:00:33.730
processes, you know, in the abstraction, I
would prefer to give you examples, and then,
00:00:33.730 --> 00:00:38.320
we would try to come to a conclusion. And
hopefully, you know, you will be able to
00:00:38.320 --> 00:00:44.370
define and in fact, you would have, by then,
by that time formed your own definition of
00:00:44.370 --> 00:00:49.620
stochastic processes. Of course, here we are
going to write, first talk about discrete
00:00:49.620 --> 00:00:54.770
stochastic processes.
Now let us just look at one example: a watch
00:00:54.770 --> 00:01:01.530
selling shop keeps a particular brand of
ladies watch; and the D i - let D i denote
00:01:01.530 --> 00:01:04.430
the demand for this brand in the i th week.
So,
00:01:04.430 --> 00:01:09.451
let us just say that our planning horizon
is 3 weeks. And so D 1 will be the demand
00:01:09.451 --> 00:01:12.790
for
this particular brand of watches in the first
00:01:12.790 --> 00:01:19.290
week; D 2 in the second week; and D 3 will
be in the third week; and then, these are,
00:01:19.290 --> 00:01:27.920
you know, D i (s) are random variables, because
the demands are not certain commodity, because
00:01:27.920 --> 00:01:35.630
otherwise, shop keeper’s job will be
very easy. So, here D i(s) are random variables
00:01:35.630 --> 00:01:37.950
and they are identically independently
00:01:37.950 --> 00:01:44.110
.distributed random variables. So, this one
simplification has been added here. So, the
00:01:44.110 --> 00:01:49.260
D
i(s) are not known; they are not certain events,
00:01:49.260 --> 00:01:52.890
but they have the same distribution and
independent; so, that means, the demand in
00:01:52.890 --> 00:01:57.429
the first week is independent of the demand
in the second week, and independent of the
00:01:57.429 --> 00:02:02.780
demand in the third week.
Let N i denote the number of watches on hand
00:02:02.780 --> 00:02:09.599
at the end of the ith week. So, let us say
by Saturday evening, the man takes stock of
00:02:09.599 --> 00:02:16.250
his things that he has - that he has on hand.
So, N i will be the same particular brand
00:02:16.250 --> 00:02:21.709
of ladies watch; he has N i of them; so, that
means, N 1 at the end of the first week; N
00:02:21.709 --> 00:02:27.170
2 at the end of second week; and N 3 at the
end of the third week. Now, orders placed
00:02:27.170 --> 00:02:36.690
for watches on Sunday evening are delivered
before the shop opens on Monday morning. So,
00:02:36.690 --> 00:02:40.860
this could be Sunday evening or
Saturday evening or whatever it is. So, before
00:02:40.860 --> 00:02:47.050
the new week begins, so on Monday
morning, before the shop opens, the watches
00:02:47.050 --> 00:02:53.760
are delivered; whatever the ordering policy.
Now, suppose the ordering policy followed
00:02:53.760 --> 00:03:01.620
by the shop owner is as follows: if no
watches in stock order for watches; that means,
00:03:01.620 --> 00:03:07.030
by Saturday evening if he realizes that he
does not have any watch of this particular
00:03:07.030 --> 00:03:14.380
brand, then he will order for 4 watches, and
they will be delivered by Monday morning.
00:03:14.380 --> 00:03:20.600
So that means, if N i is zero, order 4
watches; if N i is 1 - that means, if he has
00:03:20.600 --> 00:03:24.290
1 watch at the end of the week in stock, then
he
00:03:24.290 --> 00:03:31.790
will order for 2 watches; and finally, if
he has 2 or more watches left over by the
00:03:31.790 --> 00:03:34.840
end of
the week, then he will not order any - so,
00:03:34.840 --> 00:03:41.090
do not order. So, this is his policy.
And, of course, sales are lost when the demand
00:03:41.090 --> 00:03:45.880
exceeds the number of watches in stock.
So, if there is more demand, and you do not
00:03:45.880 --> 00:03:55.380
have that many watches, then you will lose
that - those sales; so, fine. So, now, let
00:03:55.380 --> 00:03:59.280
us look at what would be the position in the
following week.
00:03:59.280 --> 00:04:00.280
..
00:04:00.280 --> 00:04:04.260
So, N i plus 1 will be let us say… So, this
will be N i plus 1 will denote the number
00:04:04.260 --> 00:04:07.240
of
watches on hand at the end of the i plus 1
00:04:07.240 --> 00:04:11.569
th week. And how will you compute N i plus
1
00:04:11.569 --> 00:04:18.400
given N i? So, this will be, you see, if N
i is zero; that means, so this is your i th
00:04:18.400 --> 00:04:24.870
week
and this is your i plus 1 th week. So, therefore,
00:04:24.870 --> 00:04:31.740
at this point you had N i watches. Now if
N i is zero, then you ordered 4, and they
00:04:31.740 --> 00:04:35.200
were delivered by the time your i plus 1 th
week
00:04:35.200 --> 00:04:39.310
started; so, that means, then you will have
at the beginning - at this point - you will
00:04:39.310 --> 00:04:42.990
have
N i plus 4 watches, and then this demand D
00:04:42.990 --> 00:04:49.630
i plus 1; so, that means, you would meet the
demand, and then depending on whether D i
00:04:49.630 --> 00:04:55.150
plus 1 is - since N i is zero - you will
actually have 4 watches on hand and that is
00:04:55.150 --> 00:05:00.660
why I have written 4 here. So, actually, your
this thing will be 4 minus D i plus 1. And
00:05:00.660 --> 00:05:08.880
if your demand is more than this, then of
course, you will say the max of this, because
00:05:08.880 --> 00:05:11.670
he cannot have negative number of
watches.
00:05:11.670 --> 00:05:19.000
So, either you have 4 minus D i plus 1; if
D i plus 1 is less than 4 or you have no watches
00:05:19.000 --> 00:05:25.070
left at the end of the next week. So, at this
point, if you are able to meet the demand
00:05:25.070 --> 00:05:27.991
D i
plus 1 then whatever the difference, that
00:05:27.991 --> 00:05:35.090
will be the watches on hand at this point.
Otherwise, it will be zero, if D i plus 1
00:05:35.090 --> 00:05:42.190
is more than 4, right? So, similarly, if N
i is 2,
00:05:42.190 --> 00:05:50.810
what were the policies? N i is 1; this is
N i is 1. So, if N i is 1, then he orders
00:05:50.810 --> 00:05:57.230
2 watches.
So, this will be N i plus 2 minus again whatever
00:05:57.230 --> 00:06:04.550
the demand and if this number is
positive, then that will be taken as the number
00:06:04.550 --> 00:06:10.000
of watches on hand at the end of the i plus
1 th week; otherwise it will be zero. So,
00:06:10.000 --> 00:06:11.990
N i of course, you can write 1 here. So, this
is
00:06:11.990 --> 00:06:18.550
.actually it is max of 3 minus D i plus 1
comma zero. So, which ever number is positive
00:06:18.550 --> 00:06:23.860
that number you will take. So, when N i is
1, and similarly, if N i is greater than or
00:06:23.860 --> 00:06:26.430
equal
to 2, then you are not ordering any watches;
00:06:26.430 --> 00:06:33.810
2 or more you do not order. So, your
watches on hand at the beginning of the i
00:06:33.810 --> 00:06:38.320
plus 1 th week is N i and i minus D i plus
1 will
00:06:38.320 --> 00:06:43.960
be what you are left with at the end of the
i plus 1 th week. So, it will be again max
00:06:43.960 --> 00:06:44.960
of
these two.
00:06:44.960 --> 00:06:54.240
So, this is how you can… So, you see, the
situation at the end of i plus 1 th week is
00:06:54.240 --> 00:07:02.860
dependent on your situation at the end of
the i th week and the demand. So, here two
00:07:02.860 --> 00:07:09.200
random phenomena on which your state of the
system - if you can want to call it; that
00:07:09.200 --> 00:07:14.900
means, the state occupied by the system at
the i plus 1 th week is given to you by N
00:07:14.900 --> 00:07:18.360
i
plus 1 and here this is the current state.
00:07:18.360 --> 00:07:22.490
So, therefore, you can say that here your
N i plus
00:07:22.490 --> 00:07:31.400
ones are dependent on just N i and D i plus
1. So, the current demand and the state in
00:07:31.400 --> 00:07:37.690
which you were at the beginning of the i plus
1 th week. So, this is sort of trying to show
00:07:37.690 --> 00:07:45.630
you the dependence because the variables N
i plus 1 which we are trying to, you know,
00:07:45.630 --> 00:07:52.550
tell us the state of the system at the end
of every week. So, this phenomena is dependent
00:07:52.550 --> 00:07:59.259
on the two random phenomena N i and D i plus
1. So, this is one example and then we
00:07:59.259 --> 00:08:03.450
will…
So, now, I can sort of give you a definition
00:08:03.450 --> 00:08:11.160
here saying that N i(s) index by the number
of the week form a discrete stochastic process.
00:08:11.160 --> 00:08:20.210
So, then when you take these N 1, N 2, N
3 - so, these are three random variables,
00:08:20.210 --> 00:08:23.500
and they form a set. See the thing is that
you are
00:08:23.500 --> 00:08:30.759
giving them an index, which is discrete. So,
N 1, N 2, N 3 and the unit of time can be
00:08:30.759 --> 00:08:35.759
anything - here it is a week, it could be
month, it could be an hour, or whatever it
00:08:35.759 --> 00:08:39.229
is. So,
when and therefore, the discrete word. So
00:08:39.229 --> 00:08:46.670
this is a random phenomenon which is being,
you know, sort of index by discrete time period,
00:08:46.670 --> 00:08:50.680
and therefore, we would call this a
discrete stochastic process.
00:08:50.680 --> 00:08:56.820
Another example. And therefore, you may. So,
of course, this and.
00:08:56.820 --> 00:09:02.929
Now the next question to be asked is - why
study this? So, for example, I just tried
00:09:02.929 --> 00:09:06.350
to
state one or two questions that the shop owner
00:09:06.350 --> 00:09:11.509
may want to have answered, but of
course, they can be many other questions that
00:09:11.509 --> 00:09:16.730
you can also raise. So, for example, the
shop owner is interested in knowing the following
00:09:16.730 --> 00:09:24.759
- long term loss of sales due to his
reordering policy; you see, because if he
00:09:24.759 --> 00:09:28.470
can by some mechanism find out what is the
00:09:28.470 --> 00:09:34.679
.sort of estimate - may not be the exact number
- but he can estimate the number of sales
00:09:34.679 --> 00:09:40.089
that are lost, when this number is negative;
that means, he is losing out on sales
00:09:40.089 --> 00:09:44.410
whenever this number is negative. So, if there
is a mechanism by which he can find out
00:09:44.410 --> 00:09:52.480
what is his long term sales, loss of sales
due to his reordering policy, because he wants
00:09:52.480 --> 00:09:55.319
to
know whether he really has a good reordering
00:09:55.319 --> 00:10:02.199
policy or not. And then, also he may also
want to know the effects of changes he makes
00:10:02.199 --> 00:10:07.749
in his reordering policies in the reordering
policy; he may also want to change some of
00:10:07.749 --> 00:10:12.240
the orders there; and then he would want to
know would that make the situation better
00:10:12.240 --> 00:10:15.240
for him.
For example, would it reduce his long-term
00:10:15.240 --> 00:10:21.339
- say long-term word, I am using here
because, you know, it takes a while for any
00:10:21.339 --> 00:10:24.220
system to settle down; so, we will most of
the
00:10:24.220 --> 00:10:28.879
time, when we talk of any stochastic process
we want to analyze it, we would be talking
00:10:28.879 --> 00:10:34.679
about its long-term behavior. So, whatever
the disturbance is and perturbations ,they
00:10:34.679 --> 00:10:36.939
all
settle down after a while, and then you want
00:10:36.939 --> 00:10:38.970
to look at the system, because, otherwise
it
00:10:38.970 --> 00:10:45.959
will be very difficult to, you know, model
any such process, you know, when there are
00:10:45.959 --> 00:10:51.879
initially lot of tribulations or lot of perturbations,
you cannot really analyze or you cannot
00:10:51.879 --> 00:10:58.629
model such a situation. So, therefore, it
is a long-term loss of sales due to his reordering
00:10:58.629 --> 00:11:02.330
policies and then he may want to know if he
makes any changes, how will it affect his
00:11:02.330 --> 00:11:10.279
again, his revenues – essentially, he is
finally interested in revenues that he gets.
00:11:10.279 --> 00:11:16.019
Now, let us look at another example, which
is probably a simpler one. So, there is an
00:11:16.019 --> 00:11:21.689
automobile manufacturing company and has the
policy of assigning its white collar
00:11:21.689 --> 00:11:26.540
employees. So, white collar employees means
who work in their offices, office of the
00:11:26.540 --> 00:11:32.139
sales and so on, to the three sections it
has. So, the three sections it has are Production,
00:11:32.139 --> 00:11:38.379
HR - you know handling human resources, and
Sales. So, these are the three, and then
00:11:38.379 --> 00:11:43.889
see, we will now look at this model – example
- and again give you another feeling about
00:11:43.889 --> 00:11:50.399
the stochastic processes. So, the three sections
are Production, Human Resource, and
00:11:50.399 --> 00:11:51.399
Sales.
00:11:51.399 --> 00:11:52.399
..
00:11:52.399 --> 00:11:57.619
So, these are three sections in the automobile
manufacturing company where he wants to
00:11:57.619 --> 00:12:03.980
assign the white collar employees. And then,
I mean by he – I mean the owner of that
00:12:03.980 --> 00:12:11.149
manufacturing - automobile manufacturing - company;
and there is no set pattern for
00:12:11.149 --> 00:12:16.759
reassignments; at least the employees do not
know. So, there must be something in the
00:12:16.759 --> 00:12:24.309
mind of the owners how they would reassign.
So, since there are no set patterns known
00:12:24.309 --> 00:12:29.620
for the reassignments, one does not know in
which section he or she will be assigned
00:12:29.620 --> 00:12:34.139
next. So, after you have been in one section
for a while, suddenly you know that you will
00:12:34.139 --> 00:12:39.199
be transferred, but then you do not know to
which one you will be transferred. So, the
00:12:39.199 --> 00:12:44.139
next assignment may depend on the current
assignment; it is possible that wherever you
00:12:44.139 --> 00:12:49.319
are right now, it may have a bearing on where
you will be next. So, these are the kinds
00:12:49.319 --> 00:12:57.069
of… So, then, if we let X i denote the section
assign during i - the 6 month period.
00:12:57.069 --> 00:13:02.959
So, that means, now you look at one employee’s
profile suppose; just take one employee;
00:13:02.959 --> 00:13:07.470
look at his profile in the sense that you
want to keep on measuring. So, your time period
00:13:07.470 --> 00:13:12.439
is a 6 month time period; that means, when
you get assigned to a section, it is for a
00:13:12.439 --> 00:13:15.790
6
month period and then at the end of the 6
00:13:15.790 --> 00:13:21.459
month period, there will be another
reassignment to sections and you may either
00:13:21.459 --> 00:13:25.699
stay in the same section or you may get
transferred to the another one. So, any way,
00:13:25.699 --> 00:13:28.649
let X i denote the section assigned during
the
00:13:28.649 --> 00:13:35.569
i th 6 month period. And then, so the whole
process can be; that means, the whole
00:13:35.569 --> 00:13:41.290
process of the sections being assigned to
a particular employee can be described by
00:13:41.290 --> 00:13:44.649
the
sequence X 1, X 2, so on. So as long as your
00:13:44.649 --> 00:13:46.209
planning horizon you will have…
00:13:46.209 --> 00:13:51.230
.So, X 1 will tell you that in the first 6
months the particular employee is in this
00:13:51.230 --> 00:13:54.069
section
whatever the value of X 1; then X 2 will tell
00:13:54.069 --> 00:13:59.980
you the section he is in the second 6 month
period and so on. And of course, X i can take
00:13:59.980 --> 00:14:03.040
the possible values. So, let us take, let
us
00:14:03.040 --> 00:14:08.389
number the three sections. So, the first section
is Production, second section is HR Human
00:14:08.389 --> 00:14:14.569
Resource and the third is Sales. So, X i can
take three possible values whichever
00:14:14.569 --> 00:14:20.619
of the three sections. So, this will describe
to you if you like you take it up to X 10.
00:14:20.619 --> 00:14:27.709
So, that means, over the 5 years the sequence
X 1, X 2, X 3 up to X 10 will tell you the
00:14:27.709 --> 00:14:36.449
sections to which this particular employee
has been assigned. So in the discrete, so
00:14:36.449 --> 00:14:38.480
this
assignment of sections to the employee is
00:14:38.480 --> 00:14:45.199
a discrete stochastic process and it is indexed
by the periods 1, 2, 3 and so on. So now,
00:14:45.199 --> 00:14:51.670
you get the meaning. So, it is something like
the process is evolving over time and there
00:14:51.670 --> 00:14:57.619
is uncertainty about what the system, what
the state - I mean where the system - would
00:14:57.619 --> 00:15:03.319
be after, you know, each time period; one
time period is over, then where will it be
00:15:03.319 --> 00:15:10.009
next? So, therefore, there is some sort of
uncertainty about the whole process, and so
00:15:10.009 --> 00:15:12.730
this is why we are calling it a stochastic
process.
00:15:12.730 --> 00:15:20.189
Now for this particular company, an employee
may ask the following questions: if an
00:15:20.189 --> 00:15:25.489
employee is working in sales, what is the
probability that after two assignments, he
00:15:25.489 --> 00:15:29.499
will
be working in sales again? This particular
00:15:29.499 --> 00:15:36.889
employee may want an answer to this
question; or for example, if the employee
00:15:36.889 --> 00:15:43.629
is currently in Production, how many months
must pass, on the average, before he enters
00:15:43.629 --> 00:15:50.630
HR - Human Resource? So, you know, as I
said again just as for the first example,
00:15:50.630 --> 00:15:53.600
I am stating some questions; you can also
add
00:15:53.600 --> 00:15:54.600
some more.
00:15:54.600 --> 00:15:55.600
..
00:15:55.600 --> 00:15:58.679
And the third one for example, is if the employee
has been with the company for 4 years,
00:15:58.679 --> 00:16:03.009
how many times on the average he would have
been assigned to HR – to Human
00:16:03.009 --> 00:16:10.009
Resource? Then what percentage of an employee’s
assignments will be in Sales? So,
00:16:10.009 --> 00:16:15.329
these are the questions and many more.
Now why would these questions be important?
00:16:15.329 --> 00:16:20.739
Because a prospective employee, who is
going to join the company, can ask questions
00:16:20.739 --> 00:16:26.040
like this, so that he can judge about his
prospects in the company. Basically, he would
00:16:26.040 --> 00:16:29.509
like to know whether he would
professionally be satisfied with the company
00:16:29.509 --> 00:16:34.299
or not. If it turns out that he comes to know
that, you know, he will most of the time be
00:16:34.299 --> 00:16:38.850
with sales, then of course, he may not be
wanting to continue, you know, stay with the
00:16:38.850 --> 00:16:44.480
company because he may not be interested
in sales and so on. So, I am just giving an
00:16:44.480 --> 00:16:47.439
example, but there can be many such questions
that can be asked.
00:16:47.439 --> 00:16:53.749
So, the N i’s of the first example and X
i’s of the second example are not independent
00:16:53.749 --> 00:17:01.369
random variables; that you can see. In the
first example, the N i’s for the number
00:17:01.369 --> 00:17:05.130
of
particular brand of watches that were left
00:17:05.130 --> 00:17:08.150
at the end of the week that were in stock,
and
00:17:08.150 --> 00:17:13.419
so, we saw that this was dependent on what
your demand is in the following week and
00:17:13.419 --> 00:17:19.560
dependent on your ordering policies. So, you
cannot say that N 1, N 2, N 3 and so on,
00:17:19.560 --> 00:17:24.660
they are independent random variables, you
can see that there is some relationship. And
00:17:24.660 --> 00:17:31.380
similarly, for the X i’s it is possible,
see whatever the way they organizers or the
00:17:31.380 --> 00:17:32.380
owners
00:17:32.380 --> 00:17:37.361
.of the company decide to reassign the sections,
certainly where you are and how long
00:17:37.361 --> 00:17:43.810
you have been in a particular section will
have a bearing on where you will be next.
00:17:43.810 --> 00:17:47.930
So,
you can feel that these random variables are
00:17:47.930 --> 00:17:51.730
not independent. And, therefore, any kind
of
00:17:51.730 --> 00:17:57.180
computations about these random variables
will not be easy thing.
00:17:57.180 --> 00:18:05.850
So, now we will attempt to define this stochastic
process, after these two examples. So,
00:18:05.850 --> 00:18:14.271
any random process for which time can be measured
discretely and can be represented as
00:18:14.271 --> 00:18:19.930
X sequence of random variables. So, I should
add the word here - and - can be
00:18:19.930 --> 00:18:26.210
represented as a sequence of random variables;
then, this is I will call it a random process
00:18:26.210 --> 00:18:38.600
or I will call it a stochastic process; it
is called is called a stochastic process.
00:18:38.600 --> 00:18:46.510
Or we
simply - you can simply - say it is a sequence
00:18:46.510 --> 00:18:52.320
of random variables index by time. So else
stochastic process and definitely you can
00:18:52.320 --> 00:18:54.980
see that it is evolving over time, and then
you
00:18:54.980 --> 00:19:02.080
want to now look at its behavior.
And so of course, now you see that if you
00:19:02.080 --> 00:19:04.760
want to answer any of these questions that
I
00:19:04.760 --> 00:19:11.350
have posed and even on the earlier one, then
you see, you may want to know, you would
00:19:11.350 --> 00:19:16.650
need to know the joint density function of
example - if your planning horizon is 5 years,
00:19:16.650 --> 00:19:21.800
then you may want to know, you would need
to know, the joint distribution of X 1, X
00:19:21.800 --> 00:19:25.130
2
up to X 10, since they are not independent,
00:19:25.130 --> 00:19:31.650
and therefore, you cannot say that the joint
density function of X 1 to X 10 will be product
00:19:31.650 --> 00:19:35.360
of individual density function. So, you
will have to, you will need to find out, and
00:19:35.360 --> 00:19:42.320
of course, if your planning horizon is much
bigger, then you know, you can just give,
00:19:42.320 --> 00:19:47.310
you know, you can throw up your hands and
say that – no, we cannot compute joint density
00:19:47.310 --> 00:19:56.650
function of so many random variables.
So, therefore, we need to really look at the
00:19:56.650 --> 00:20:05.300
methods by which we can sort of simplify
analyzing such process, or under what conditions
00:20:05.300 --> 00:20:13.260
can we try to answer questions like this
when we are looking at a stochastic process.
00:20:13.260 --> 00:20:14.260
..
00:20:14.260 --> 00:20:23.531
So, for the automobile company, just look
at the… we can diagrammatically describe
00:20:23.531 --> 00:20:28.740
the
profile of an employee, and so you see, here
00:20:28.740 --> 00:20:35.300
the horizontal axis is giving you the time
period. So, this is the beginning of the planning
00:20:35.300 --> 00:20:40.270
horizon - so zero period, that means, the
start of the process; then, this denotes the
00:20:40.270 --> 00:20:42.880
first 6 months period; this is the second
6
00:20:42.880 --> 00:20:46.830
months period; third 6 months period and so
on. So this is, what it is.
00:20:46.830 --> 00:20:52.170
And then, here you have the three sections
to which the person can be assigned. So, for
00:20:52.170 --> 00:20:57.800
example, what it is saying is that here in
the first 6 month period he was with HR - the
00:20:57.800 --> 00:21:02.970
second section; and then in the next 6 month
period he got assigned to Production, that
00:21:02.970 --> 00:21:05.920
is
your first section; I think this is Production,
00:21:05.920 --> 00:21:09.200
this is HR and this is Sales. So, then he
got
00:21:09.200 --> 00:21:14.610
assigned to in the second 6 month period he
went to Sales; and then again after that,
00:21:14.610 --> 00:21:20.800
he
went to sales in the next this month; that
00:21:20.800 --> 00:21:23.700
means, 1 year is over and this is the next
6
00:21:23.700 --> 00:21:29.290
months in it. So, therefore, you can see that
this diagram, and here for example, here
00:21:29.290 --> 00:21:36.170
from this onwards he continued for two periods
consecutively in the Production section.
00:21:36.170 --> 00:21:44.440
So, this you can diagrammatically describe
the profile of an employee in the
00:21:44.440 --> 00:21:50.280
manufacturing company.
Now let us just give some more terminology.
00:21:50.280 --> 00:21:58.580
So, set of all possible values the random
variable X i takes is called the state space.
00:21:58.580 --> 00:22:05.640
So, we always describe, so when whenever
the system or the process is, whatever situation
00:22:05.640 --> 00:22:10.600
it is in, so that would be described by the
state space and normally what we do is we
00:22:10.600 --> 00:22:11.600
give it numbers.
00:22:11.600 --> 00:22:16.690
.So, the possible values in the state space
we describe by the numbers. So, for example,
00:22:16.690 --> 00:22:21.290
here the three sections I have numbered as
1, 2, 3. So, it is easier; because, otherwise,
00:22:21.290 --> 00:22:27.270
you cannot go writing the possible values
that the state space contains; it may different,
00:22:27.270 --> 00:22:33.180
different things. So, we can just distinguish
by the numbers. And so here, for example,
00:22:33.180 --> 00:22:39.680
this will be X n is i, means the state in
which the system is at time period N. So,
00:22:39.680 --> 00:22:43.630
the
value of X n - so, if I am describing the
00:22:43.630 --> 00:22:50.760
my X i’s are the variables - which are the
random variables - which describe the process.
00:22:50.760 --> 00:22:56.390
And then, when you change these to, that
means, when the system changes from one state
00:22:56.390 --> 00:23:04.350
to another, we call such process as - the
change it is called as transition or transitions.
00:23:04.350 --> 00:23:11.620
Now as I said that and we have seen the two
examples already, simple ones. We saw that
00:23:11.620 --> 00:23:17.950
the real life situations the processes will
be many; many, many processes that are
00:23:17.950 --> 00:23:26.180
stochastic because there are elements of the
process which cannot be determined with
00:23:26.180 --> 00:23:31.580
certainty, and then we have also seen that,
you know, even in such simple examples your
00:23:31.580 --> 00:23:37.620
X i’s are not independent. So, there will
be some sort of dependence among the random
00:23:37.620 --> 00:23:41.910
variables.
So, therefore, as I was saying earlier, that
00:23:41.910 --> 00:23:47.100
it will be very difficult to have a combined
joint density function of all the possible
00:23:47.100 --> 00:23:50.360
random variables which describe the states
in
00:23:50.360 --> 00:23:56.500
which the system can be over long time period.
And so, you cannot just analyze or
00:23:56.500 --> 00:24:01.680
answer any questions about the process.
So, Markov suggested the following simplification.
00:24:01.680 --> 00:24:07.650
So, he said that the transition from
one section to another may depend on the current
00:24:07.650 --> 00:24:13.300
section occupied and here I should say
the word only, the transition from one section
00:24:13.300 --> 00:24:18.100
to another may depend on the current
section occupied. So, when we say depend,
00:24:18.100 --> 00:24:20.030
this of course, that means, the computation
of
00:24:20.030 --> 00:24:26.600
the probability, the probability with which
the process will transition from one section
00:24:26.600 --> 00:24:32.760
to
another, the probability would be dependent
00:24:32.760 --> 00:24:38.500
on where you are right now - so, the current
section. So Markov suggested this simplification.
00:24:38.500 --> 00:24:44.750
And for example, in the watch shop example
value of N i plus 1 to depend on the values
00:24:44.750 --> 00:24:51.250
of N i and D i plus 1 only. And the way I
was describing to you the values of N i plus
00:24:51.250 --> 00:24:54.840
1
which was max of the formulae I wrote down.
00:24:54.840 --> 00:24:59.250
So that, from there we saw that we were
computing N i plus 1 only depending on the
00:24:59.250 --> 00:25:02.410
values of N i and D i plus 1. So, that was,
00:25:02.410 --> 00:25:10.540
.that was anyway according to Markov’s definition;
this is already satisfying the
00:25:10.540 --> 00:25:13.200
Markovian property.
.
00:25:13.200 --> 00:25:23.660
So, when once you have this, and then that
means, in the section assignment problem
00:25:23.660 --> 00:25:30.560
what we are saying is that - we are saying
that if you want to look at X i - the value
00:25:30.560 --> 00:25:33.620
of X
i - then that will depend on… so the probability
00:25:33.620 --> 00:25:38.990
that you will go from, whatever the
value of X i, it will depend on what is the
00:25:38.990 --> 00:25:44.200
value of X i minus 1. So, sort of the transition
from here; so this will depend on this, and
00:25:44.200 --> 00:25:48.910
then X i will affect the value of X i plus
1
00:25:48.910 --> 00:25:53.920
with certain probability. So, this is the
kind of dependence we are only allowing or
00:25:53.920 --> 00:25:56.800
you
can say this the simplification.
00:25:56.800 --> 00:26:03.080
So, this makes the analysis of stochastic
processes, which satisfies Markov’s property
00:26:03.080 --> 00:26:09.170
quite tractable, and we will see this, as
we go on, we will see that we can probably
00:26:09.170 --> 00:26:14.910
answer almost all the questions that I wrote
in the beginning, about the automobile
00:26:14.910 --> 00:26:19.540
manufacturing company and the question that
- the kind of questions - that an employee
00:26:19.540 --> 00:26:24.910
may be interested in knowing. So, we should
be, we will be able to answer the questions,
00:26:24.910 --> 00:26:34.370
because if we say that the section assignment
process would satisfy the Markov property.
00:26:34.370 --> 00:26:39.770
Now any stochastic process, which satisfies
Markov’s property, is called a Markov chain
00:26:39.770 --> 00:26:44.700
or a Markov process. So, I will be using the
word Markov chain or Markov process with
00:26:44.700 --> 00:26:47.120
the same meaning - synonymously.
00:26:47.120 --> 00:26:54.250
.So, now, what happens is that with the Markov’s
property being satisfied by the process,
00:26:54.250 --> 00:26:58.690
then we just need to compute the joint or
the conditional probability mass function;
00:26:58.690 --> 00:27:04.140
remember I am talking about discrete processes.
So, joint or conditional P M F of
00:27:04.140 --> 00:27:10.010
neighboring X i’s is computed. So, it simplifies
and therefore, you know, we are having
00:27:10.010 --> 00:27:15.360
two variables, you can very easily compute
the joint or the conditional P M F of two
00:27:15.360 --> 00:27:23.230
variables. And so, with that, we are then
able to analyze the process over long term
00:27:23.230 --> 00:27:24.310
and
whatever it is.
00:27:24.310 --> 00:27:32.560
So, now if you want to formally state Markov’s
property, that is, see essentially what you
00:27:32.560 --> 00:27:39.070
are saying is - probability X n plus 1 is
equal to j; that means, at time N plus 1 your
00:27:39.070 --> 00:27:46.309
system is occupying state j and if you look
at the past history starting from X 0 is i,
00:27:46.309 --> 00:27:49.710
then
it will be X 1 is some i 1 and so on. X n
00:27:49.710 --> 00:27:55.710
minus 1 is i n minus 1 and X n is i. So, this
is the
00:27:55.710 --> 00:28:01.620
entire past history. So, if you are not assuming
the Markov property being satisfied by
00:28:01.620 --> 00:28:06.530
the process, then of course, to answer this
- compute this probability - you would need
00:28:06.530 --> 00:28:09.610
to
know the entire past history, but then, Markov’s
00:28:09.610 --> 00:28:15.680
property simplifies it and says, that this
whole thing can be made equal to probability
00:28:15.680 --> 00:28:19.260
X n plus 1 equal to j given that X n is i;
so
00:28:19.260 --> 00:28:23.860
whatever the condition; so the current state
of a system that helps you to determine, so
00:28:23.860 --> 00:28:30.940
with some probability where the system will
be in the next state, next time period. And
00:28:30.940 --> 00:28:37.650
so, these are known as one-step transitional
probabilities and we will call them as p i
00:28:37.650 --> 00:28:40.930
j.
So, now here I am not writing anything else,
00:28:40.930 --> 00:28:45.930
why because I am now making one more
simplification; and what we are saying is
00:28:45.930 --> 00:28:51.110
that, this is actually equal to probability
of X 1
00:28:51.110 --> 00:28:57.190
equal to j given that X naught is i; so, that
means, the starting probability, the starting
00:28:57.190 --> 00:29:03.080
state of the system, suppose if you were in
system was in i - state i - then the next
00:29:03.080 --> 00:29:07.660
period
that it is in j. So, we will denote that one-step
00:29:07.660 --> 00:29:13.700
transitional probability and we will say that
over the long period that the process goes
00:29:13.700 --> 00:29:20.230
on, this does not change; that means, whether
at the time period N plus 1 you are considering
00:29:20.230 --> 00:29:25.860
the change from X time period N to time
period N plus 1 or you are considering the
00:29:25.860 --> 00:29:30.650
change from the starting - initial state - to
the
00:29:30.650 --> 00:29:35.400
first period. So, those probabilities remain
the same and that is why the word stationary.
00:29:35.400 --> 00:29:39.830
So, what we are saying is that the one-step
transitional probabilities are stationary.
00:29:39.830 --> 00:29:42.590
And
essentially the explanation here is that whatever
00:29:42.590 --> 00:29:48.230
process you consider, we are saying that
after the initial perturbations and so on,
00:29:48.230 --> 00:29:53.200
this system has settled down to stationary;
this
00:29:53.200 --> 00:29:58.360
system has become - or the process has become
- stationary. So, it is not, and therefore,
00:29:58.360 --> 00:30:04.680
these transitional probabilities are not being
affected by where you are considering - at
00:30:04.680 --> 00:30:11.750
.what time period you are considering the
transition. As we go on, we will be looking
00:30:11.750 --> 00:30:15.140
at a
lot of processes and lot of situations - real
00:30:15.140 --> 00:30:21.340
life situations - where we will see that to
assume that we have transitional probabilities
00:30:21.340 --> 00:30:27.830
have the stationary property is not very
unrealistic. So, we will continue with the…I
00:30:27.830 --> 00:30:33.880
will just continue with defining and giving
you how to compute these probabilities and
00:30:33.880 --> 00:30:40.010
so on. Once you have these probabilities,
then what can you do with these?
00:30:40.010 --> 00:30:41.010
.
00:30:41.010 --> 00:30:47.690
So, let us start looking at how we will now
continue with the analysis of the process.
00:30:47.690 --> 00:30:51.210
So,
therefore, what we would need first to describe
00:30:51.210 --> 00:30:56.500
the process, and then, what are the
quantities that we would require before we
00:30:56.500 --> 00:30:59.110
can continue with our analysis and trying
to
00:30:59.110 --> 00:31:04.990
answer the questions related to the process.
So, if X naught is the… let us say X naught
00:31:04.990 --> 00:31:09.720
might be the present assignment by our
notation zero. So, this means, whatever value
00:31:09.720 --> 00:31:14.030
of X naught, that tells us the present
assignment of the employees, and then, we
00:31:14.030 --> 00:31:17.610
are interested in his next assignment. That
is,
00:31:17.610 --> 00:31:24.350
we want to know the value of X 1 in the next
time period. So, if suppose X naught is 1,
00:31:24.350 --> 00:31:31.030
that is the man is currently in Production,
then X 1 can be 1, 2, or 3 any of the three
00:31:31.030 --> 00:31:38.301
sections he can be assigned to.
So, that means, you would want to know what
00:31:38.301 --> 00:31:40.210
the probability is. So, it means this has
to
00:31:40.210 --> 00:31:48.330
be given to you, that is if he is already
in, he is starting his career with X naught
00:31:48.330 --> 00:31:51.250
equal to
1; that means, he is in Production right now;
00:31:51.250 --> 00:31:53.179
and then, what is the probability that he
will
00:31:53.179 --> 00:31:59.490
.be again kept in Production only? So, X 1
is 1. So, we will call this as a P 1 1. And
00:31:59.490 --> 00:32:03.490
as I
told you these are one-step transitions probabilities
00:32:03.490 --> 00:32:08.830
and that we are assuming stationality.
So, it does not matter whether it is X n plus
00:32:08.830 --> 00:32:14.160
1 equal to 1 given X n is 1; or X naught is
1
00:32:14.160 --> 00:32:19.670
given that X naught is 1 then X n is 1, so
the probability. So, P 1 1. And then, similarly,
00:32:19.670 --> 00:32:25.450
you would need to know if X naught is 1, then
what is the probability that he will be in
00:32:25.450 --> 00:32:32.600
HR? So, that probability is P 1 2 and the
probability that he will be in Sales is given
00:32:32.600 --> 00:32:36.210
by P
1 3. So, these are the first step transitional
00:32:36.210 --> 00:32:41.620
probabilities.
If you know where he is at the beginning of
00:32:41.620 --> 00:32:49.190
the planning horizon. Again if you know X
naught is 1, but if X naught is 2, then of
00:32:49.190 --> 00:32:50.710
course, again their transitional probabilities
will
00:32:50.710 --> 00:32:55.760
be different - in the sense that now what
is this probability of going from 2 to 1?
00:32:55.760 --> 00:32:59.270
So that
means, he is in HR and then the probability
00:32:59.270 --> 00:33:03.930
that he will be assigned to production; so,
that must be some probability. You see these
00:33:03.930 --> 00:33:09.309
are the transition probabilities, which are
now describing to us whatever the assignment
00:33:09.309 --> 00:33:16.160
process is.
So, therefore, again these three transition
00:33:16.160 --> 00:33:19.750
probabilities P 2 1, P 2 2 and P 2 3 are given
to
00:33:19.750 --> 00:33:25.660
us; and then for if X naught is equal to 3,
that means if he is already in, he is currently
00:33:25.660 --> 00:33:29.270
in
Sales, then his probability of going into
00:33:29.270 --> 00:33:35.620
Production will be P 3 1, probability of going
into Sales will be P 3 2, and probability
00:33:35.620 --> 00:33:39.390
of remaining and Sales will be given by P
3 3.
00:33:39.390 --> 00:33:44.600
So, these we call as the… I am not all the
time saying one-step transition probabilities,
00:33:44.600 --> 00:33:51.340
but that is understood. So this is transitioning
from state. So here, these three numbers these
00:33:51.340 --> 00:33:55.760
three probabilities - give you the transition
probabilities of transitioning from state
00:33:55.760 --> 00:34:03.420
3 to any of the three states. So, therefore,
the process, and now, of course, we will see
00:34:03.420 --> 00:34:09.669
that this is not a complete description of
the process, and we will, as we go along we
00:34:09.669 --> 00:34:12.619
will
find out what more we need; but, let us first
00:34:12.619 --> 00:34:17.260
just look at this.
So, now, the nine first step transitional
00:34:17.260 --> 00:34:20.570
probabilities can also be written as 3 by
3. See,
00:34:20.570 --> 00:34:25.510
remember, because whatever the number of states,
if the number of states is capital N,
00:34:25.510 --> 00:34:31.759
then your transition probabilities will be
N square, because you can go from one state
00:34:31.759 --> 00:34:34.659
to
any of the N states. So, therefore, you will
00:34:34.659 --> 00:34:40.450
always have N square numbers. And so, these
transition probabilities can be written in
00:34:40.450 --> 00:34:45.099
a matrix form. So, if you have N states that
this
00:34:45.099 --> 00:34:51.220
system can occupy, then it will be N cross
N matrix that you can you can record all these
00:34:51.220 --> 00:34:57.230
transitional probabilities in an N by N matrix.
So, here since our states, 3 states are there,
00:34:57.230 --> 00:35:02.540
three sections. So, I can record all the nine
transitions - first step transition - probabilities
00:35:02.540 --> 00:35:08.390
and then 3 by 3 matrix. So, P will be called
transition matrix.
00:35:08.390 --> 00:35:18.359
.Now since the man must transition from, let
us say from, Production to any one of these
00:35:18.359 --> 00:35:24.789
other, either he stays in Sales, Production
or he goes to HR or he goes to Sales, he must
00:35:24.789 --> 00:35:30.700
transition to one of them, because after every
6 months the assignment is announced.
00:35:30.700 --> 00:35:31.700
.
00:35:31.700 --> 00:35:44.140
So, therefore, these three probabilities will
add up to 1. And therefore, also another way
00:35:44.140 --> 00:35:49.920
of saying that these probabilities must add
up to 1 is that they are the P 1 1 and P 1
00:35:49.920 --> 00:35:53.220
2 and
P 1 3 describe the conditional P M F; remember
00:35:53.220 --> 00:35:58.940
we have talked about it while talking
about, you know, conditional probabilities
00:35:58.940 --> 00:36:03.420
and conditional expectations.
So, this is the conditional P M F of X 1 given
00:36:03.420 --> 00:36:09.619
that X naught is 1. And, so therefore, since
these three numbers describe the conditional
00:36:09.619 --> 00:36:15.250
P M F they must add up to 1 because of
this. In the same way you can argue that the
00:36:15.250 --> 00:36:20.869
second and the third rows must also add up
to 1; that means, P 2 1 plus P 2 2 plus P
00:36:20.869 --> 00:36:26.200
2 3 is equal to 1 and P 3 1 plus P 3 2 plus
P 3 3
00:36:26.200 --> 00:36:31.369
is 1. So, now, any square matrix which has
- because these are probabilities, so they
00:36:31.369 --> 00:36:34.140
have
to be nonnegative numbers - so any matrix
00:36:34.140 --> 00:36:37.380
- a square matrix - which has all entries
or all
00:36:37.380 --> 00:36:43.839
elements nonnegative and the rows add up to
1, can qualify to be a transition matrix;
00:36:43.839 --> 00:36:47.239
that
means, we can say that there must be a stochastic
00:36:47.239 --> 00:36:52.380
process, which can be associated with
such a matrix. So, all entries are nonnegative
00:36:52.380 --> 00:37:00.039
and the rows - the elements of row - add up
to 1 - of every row - add up to 1. So, this
00:37:00.039 --> 00:37:03.920
will be - this will qualify to be a transition
matrix.
00:37:03.920 --> 00:37:09.089
.Now, another way of looking at this process,
because diagrams always help, they fix
00:37:09.089 --> 00:37:17.509
ideas, and I think they also help in the understanding
of the process. So, let us see, I will
00:37:17.509 --> 00:37:24.630
describe the three states by the nodes of
this graph. So first is your Production, HR
00:37:24.630 --> 00:37:29.320
and
Sales. And if I am showing it arc from 1 to
00:37:29.320 --> 00:37:33.160
2, then this is transitioning from 1 to 2;
and of
00:37:33.160 --> 00:37:39.299
course, I have not entered all the probabilities,
but they can be written down here. And
00:37:39.299 --> 00:37:47.700
so, the arc 2 to 1 will be the transition
from - one-step transition - from your HR
00:37:47.700 --> 00:37:52.039
to
Production. And this loop describes, that
00:37:52.039 --> 00:37:56.220
means, you stay in one, that means, your
transition from one to one. So, you do not
00:37:56.220 --> 00:37:59.440
go anywhere, you continue with the same
state.
00:37:59.440 --> 00:38:05.289
So, this way you can look at this. And so,
you can write down the probabilities. Here
00:38:05.289 --> 00:38:10.250
also
P 1 1; this will be P 2 2; and this will be
00:38:10.250 --> 00:38:17.489
P 2 3 and this will be P 3 2, and finally,
this will
00:38:17.489 --> 00:38:27.069
also be P 3 3, and here this will be P 3 1
and this will be P 1 3. So, this diagram also
00:38:27.069 --> 00:38:32.670
helps you to look at… and you can see that
currently you can transition from for
00:38:32.670 --> 00:38:38.099
example, from 1 to 2, then you can go from
2 to 3, you can come from 3, again you can
00:38:38.099 --> 00:38:42.990
come back to 2 or you can go from 2 to 2.
So, actually, you can play around and you
00:38:42.990 --> 00:38:46.200
can
see lot of things you can do with the… you
00:38:46.200 --> 00:38:49.960
can see how the transition is taking place
and
00:38:49.960 --> 00:38:54.480
so on. But, of course, this you can do when
your number of states is small.
00:38:54.480 --> 00:39:01.550
And if the number of states is large, then
drawing a picture like this may not be a very
00:39:01.550 --> 00:39:08.739
good alternative, and so, we will have to
look at other ways of handling this process;
00:39:08.739 --> 00:39:12.430
but
anyway, this makes the thing look interesting;
00:39:12.430 --> 00:39:17.460
I mean the picture is there and you can
just see how the process is evolving over
00:39:17.460 --> 00:39:23.710
time going from one state to another and going
through these parts. So, you see we describe
00:39:23.710 --> 00:39:26.690
the first step transitional probabilities
and
00:39:26.690 --> 00:39:28.320
through a diagram and so on.
00:39:28.320 --> 00:39:29.320
..
00:39:29.320 --> 00:39:34.369
Now, suppose we want to now look at look at
X 2 - the random variable that describes
00:39:34.369 --> 00:39:40.289
the state of the system at time 2. Now, again
I will try to show you through the diagram.
00:39:40.289 --> 00:39:45.460
So, if you look at this for example, you start
with state one; that means, you start with
00:39:45.460 --> 00:39:51.380
Production and after two transitions you are
back in Production. So, what would that
00:39:51.380 --> 00:39:57.369
mean? So, the possibilities are that you start
with Production, then the next period you
00:39:57.369 --> 00:40:01.799
again transition to Production only; that
means, you stay where you are; and then finally,
00:40:01.799 --> 00:40:06.829
you, in the next step you again transition
to Production. So, that means, you continue
00:40:06.829 --> 00:40:12.040
through this. So, this path describes one
possibility which I have written down here.
00:40:12.040 --> 00:40:17.539
And then, it could be that you start from
Production, you go to HR, and then again,
00:40:17.539 --> 00:40:20.749
you
get transition back to Production. So, that
00:40:20.749 --> 00:40:25.599
would be your second path. So, I am talking
now in terms of paths; because this is how
00:40:25.599 --> 00:40:30.200
you will - when you go for two-step transition
probabilities - this is what you will have
00:40:30.200 --> 00:40:32.930
to do - compute the probabilities of these
paths.
00:40:32.930 --> 00:40:38.479
And then finally, you will start from Production,
go to Sales, and then you are back to
00:40:38.479 --> 00:40:43.740
Production and that will be your third path.
So, just to give you, and of course, we will
00:40:43.740 --> 00:40:50.790
continue this discussion in next lecture also.
So if you look at probability X 2…So, here
00:40:50.790 --> 00:40:58.900
your the arrows are in the wrong direction.
So, it should be X naught to 1 and then X
00:40:58.900 --> 00:41:03.809
1 to X 2. You start from here, then you
transition to Production, and again you transition
00:41:03.809 --> 00:41:08.930
to Production from the first period to
the second period. And since we have the Markovian
00:41:08.930 --> 00:41:10.529
property that tells us, that, you
00:41:10.529 --> 00:41:17.829
.know, we just need one-step transition probabilities,
that means, the transitioning from X
00:41:17.829 --> 00:41:22.880
naught 1 to 1 and then from in the second
period, from in the first period from 1 to
00:41:22.880 --> 00:41:25.229
this;
these are independent, and therefore, I can
00:41:25.229 --> 00:41:32.069
write them as the product of transition
probability 1 to 1 here in the first period,
00:41:32.069 --> 00:41:36.229
and then again 1 to 1.
So, now here again the second property that
00:41:36.229 --> 00:41:42.269
we have used is the stationality. So,
Markovian property and stationary transition
00:41:42.269 --> 00:41:48.170
probabilities both tell us that, you know,
the probability of the first path; that means,
00:41:48.170 --> 00:41:55.339
transitioning from 1 to 1 in 2 periods along
the first path, the probability is P 1 1 square.
00:41:55.339 --> 00:41:59.420
And so, we will continue with this kind of
computation, and then show you very interesting
00:41:59.420 --> 00:42:06.880
results. And then, you will see that how
far our analysis can go of a stochastic process
00:42:06.880 --> 00:42:19.829
which satisfies Markovian property and of
course, we are talking when the stationality
00:42:19.829 --> 00:42:21.589
conditions are met.
00:42:21.589 --> 00:42:22.589
.