WEBVTT
00:14.790 --> 00:21.790
Good morning, and welcome to this the lecture
number 4, of the course, Water Resources System
00:22.039 --> 00:29.039
- Modeling Techniques and Analysis. In the
previous lecture, we have seen for functions
00:30.860 --> 00:33.320
of single variables.
00:33.320 --> 00:40.320
How we pose the necessary conditions for the
optimum values, and we were just about to
00:42.910 --> 00:47.980
start with the sufficiency conditions in the
previous lecture. So, just look at the summary
00:47.980 --> 00:54.980
of what we covered in the previous lecture;
we started with the distinction between optimization
00:55.150 --> 01:01.970
and simulation, in optimization we are looking
for maximum or minimum value of a function.
01:01.970 --> 01:08.220
Whereas, in the simulation we are essentially
trying to mimic the behavior of a particular
01:08.220 --> 01:14.720
system, and simulation is also a very powerful
technique especially when we are looking at
01:14.720 --> 01:21.720
screening of alternatives in large river basins
and so on. Whereas, optimization why you used
01:22.250 --> 01:29.250
to get the optimal values of a system performance.
Let us say we are talking about optimum hydro
01:31.729 --> 01:37.610
power development or optimum flood control
and so on. So, in situations where you are
01:37.610 --> 01:43.360
really interested in the best values that
you can derive out of the system you use optimization.
01:43.360 --> 01:48.500
However in many situations, as I mentioned
we may not be interested in single optimum
01:48.500 --> 01:55.500
values, but we may be interested in answering
questions such as what if types of questions,
01:57.000 --> 02:04.000
in which simulation will be a powerful technique.
Then we went on to examine functions of single
02:05.689 --> 02:12.689
variables, and specifically we talked about
local maximum, local minimum, and then the
02:12.700 --> 02:18.310
saddle point where the first slope of the
function is 0, yet it may not correspond to
02:18.310 --> 02:25.310
either a maximum or a minimum value, and then
we have a seen the definitions of a convex
02:25.409 --> 02:32.120
function and concave function; the in the
case of convex function the local minimum
02:32.120 --> 02:37.799
also corresponds to the global minimum, in
the in the case of concave function - the
02:37.799 --> 02:44.799
global the local maximum also corresponds
to the global maximum. So, this is what we
02:44.859 --> 02:51.709
covered in the previous lecture. Now, we will
start with the optimization of functions of
02:51.709 --> 02:58.709
a single variable and post on necessary and
sufficiency conditions more formally.
02:58.909 --> 03:05.909
So, you have a function f of x as function
of a single variable. Now for this function
03:09.529 --> 03:15.779
to have an optimum at a particular point x
is equal to x naught; the necessary condition
03:15.779 --> 03:21.169
as I have been mentioning is that the slope
of that function at that particular point
03:21.169 --> 03:26.790
x is equal to x naught must be 0; so, the
slope must be 0 that is an necessary condition.
03:26.790 --> 03:33.790
So, at a local optimum which is either a maximum
or a minimum f dash of x is equal to 0. So,
03:34.769 --> 03:40.719
typically what we do is given the function
you obtain f dash of x it is the first derivative
03:40.719 --> 03:47.719
set it to 0, and solve for x; you may get
many several solutions are x 1, x 2, x 3 etcetera,
03:50.589 --> 03:57.499
all of its satisfy this f dash of x is equal
to 0. And that we define you also stationary
03:57.499 --> 04:02.079
prime; so, x is equal to x naught is the stationary
point which satisfies f dash of x is equal
04:02.079 --> 04:08.109
to 0.
Now, this is a necessary condition; for f
04:08.109 --> 04:15.109
of x to have a minimum or a maximum at the
stationary point x is equal to x naught, the
04:15.479 --> 04:22.479
sufficiency conditions are f double dash of
x evaluated at x naught less than 0, if f
04:24.630 --> 04:30.710
double dash of x which is the second derivative
of the function evaluated at x is equal to
04:30.710 --> 04:37.420
x naught, if it is negative then the point
x is equal to x naught - x naught corresponds
04:37.420 --> 04:44.420
to a maximum value. If f double dash of x
evaluated at x is equal to naught is greater
04:45.380 --> 04:52.380
than 0 which is positive, then the point x
is equal to x naught corresponds to a minimum
04:52.450 --> 04:58.110
value which means the function will have a
minimum value at x is equal to x naught. So,
04:58.110 --> 05:04.540
these are the sufficiency conditions. So,
the necessary condition is that the slope
05:04.540 --> 05:09.430
must be 0 at that point.
So, what you do is you set f dash of x is
05:09.430 --> 05:14.510
equal to 0 solve for x is equal to x naught,
you get the stationary point, you may not
05:14.510 --> 05:19.780
get just one solution you may you may get
2, 3, 4, etcetera depending on the nature
05:19.780 --> 05:26.780
of f dash of x. At a given x is equal to x
naught, you go to the second order derivative,
05:29.600 --> 05:35.710
evaluate the second order derivative at x
is equal to x naught, and then examine whether
05:35.710 --> 05:41.930
f double dash of x is less than 0 at x is
equal to x naught, if it is less than 0 at
05:41.930 --> 05:48.580
x is equal to x naught the point x is equal
to x naught corresponds to a maximum value,
05:48.580 --> 05:54.700
if f double dash of x evaluated at x is equal
to x naught is greater than 0 or it is positive,
05:54.700 --> 06:01.030
then the point x is equal to x naught corresponds
to a minimum value.
06:01.030 --> 06:08.030
Now, what if f double dash of x evaluated
at x is equal to x naught is also 0; that
06:09.520 --> 06:14.200
means, we started with f dash of x is equal
to 0, solve for x is equal to x naught, solve
06:14.200 --> 06:21.070
for x naught, and then evaluated the second
order derivative at x is equal to x naught;
06:21.070 --> 06:28.070
now, what if f double dash of x evaluated
at x is equal to x naught is also 0. In such
06:29.090 --> 06:35.900
case you go to the higher order derivative,
get f triple dash x naught, x f triple dash
06:35.900 --> 06:41.430
x at x is equal to x naught; that means, the
third derivative third order derivative evaluated
06:41.430 --> 06:48.430
at x is equal to x naught; what if this is
also 0, go to fourth order derivative, evaluate
06:49.390 --> 06:54.290
at x is equal to x naught; what if that is
also 0 go to fifth order derivative, evaluate
06:54.290 --> 07:01.140
at x is equal to x naught; what if that is
also 0 and so on. You keep continuing until
07:01.140 --> 07:08.140
you get the first non-zero derivative, evaluated
at x is equal to x naught.
07:10.740 --> 07:17.740
And then, you look at the order of the derivative.
I will state this more formally. So, we use
07:18.170 --> 07:23.770
the second order derivative our sufficiency
condition, if the second order derivative
07:23.770 --> 07:29.560
is also 0, go to the third order derivative,
fourth order derivative, fifth order derivative
07:29.560 --> 07:36.560
and so on, until you get the first derivative
evaluated at x is equal to x naught which
07:36.780 --> 07:43.520
is not not zero. And then look at the order
of the derivative at which you are getting
07:43.520 --> 07:46.340
a non-zero derivative value.
07:46.340 --> 07:53.340
So, if the second order derivative is 0, find
the first higher order non-zero derivative.
07:54.090 --> 07:59.360
Let this be the n th order derivative, what
I mean by that is d f by d x is equal to d
07:59.360 --> 08:05.389
square of f by d x square d q f by d x cube
etcetera n minus 1 n th order derivative they
08:05.389 --> 08:12.389
are all 0, and the n th order derivative is
the first non-zero derivative that you are
08:12.669 --> 08:19.669
getting when you are evaluating the derivative
at x is equal to x naught. Then you look at
08:20.690 --> 08:27.690
the order of the derivative; if the order
of the derivative is even, and this value
08:30.050 --> 08:36.889
of this derivative that you get is negative,
then it the point corresponds to a maximum
08:36.889 --> 08:43.889
value. If the order of the derivative n is
the; so,… So, obtained is even and the derivative
08:47.329 --> 08:54.269
value is positive, then the value the point
x is equal to x naught corresponds to a minimum.
08:54.269 --> 09:01.269
Remember always here, the higher order derivative
negative always corresponds to maximum; higher
09:02.629 --> 09:06.610
derivative positive always corresponds to
minimum. This you must keep in mind. So, the
09:06.610 --> 09:12.290
same thing we apply you keep going higher
and higher orders of the derivative, until
09:12.290 --> 09:19.290
you get the first derivative which is nonzero.
Then, you look at the order of the derivative;
09:20.589 --> 09:27.589
if that order is even and the magnitude of
the derivative is negative, then it corresponds
09:29.420 --> 09:36.420
to maximum value. If the order is even and
the magnitude is odd, I am sorry and the magnitude
09:38.069 --> 09:45.069
is positive then the value corresponds to
a minimum value.
09:45.980 --> 09:52.980
If the first derivative which is non-zero,
and the order of that non-zero derivative
09:56.860 --> 10:03.860
is odd; then the point x is equal to x naught
neither corresponds to a minimum value nor
10:07.009 --> 10:14.009
corresponds to a maximum value. I repeat this
again before; so, formally stating it; we
10:17.269 --> 10:21.829
use the second order derivative as a sufficiency
condition, if the second order derivative
10:21.829 --> 10:27.519
is negative then the point x is equal to x
naught corresponds to a maximum value. If
10:27.519 --> 10:32.339
the second order derivative is positive then
the point x is equal to x naught corresponds
10:32.339 --> 10:37.749
to a minimum value.
If the second order derivative is 0 then we
10:37.749 --> 10:43.180
go to higher order derivative, third order
derivative; if it is also 0 go to fourth order
10:43.180 --> 10:49.499
derivative that is also 0 go to fifth order
derivative and so on. Get the first derivative
10:49.499 --> 10:56.499
which is non-zero, then you look at the order
of the derivative, let us say this was sixth
10:58.089 --> 11:04.910
order derivative which was non-zer0. The order
of derivative is 6 which is a even number.
11:04.910 --> 11:11.910
So, if the first non-zero derivative is of
the order which is a even order, then if the
11:15.730 --> 11:21.639
magnitude of the derivative is negative then
the point x is equal to x naught which is
11:21.639 --> 11:27.999
stationary point corresponds to a maximum
value. If the magnitude is positive then the
11:27.999 --> 11:34.999
point x is equal to x naught corresponds to
a minimum value; if however, the order of
11:35.889 --> 11:42.889
the derivative is odd then the point x is
equal to x naught corresponds neither to a
11:43.399 --> 11:49.829
minimum value nor to a maximum value. So,
this what we state formally.
11:49.829 --> 11:56.829
So, if n is odd, and d n f by d x n which
is the n th order of derivative of the function
11:59.499 --> 12:06.499
evaluated at x is equal to x naught is positive,
then x naught is a local minimum. This is
12:07.569 --> 12:13.220
a more informal way of putting it what we
mean by that is x naught corresponds to the
12:13.220 --> 12:19.389
function having a local minimum at that point
or the function has a local minimum at x is
12:19.389 --> 12:26.389
equal to x naught. If n is even and d n f
by d x n evaluated at x is equal to x naught
12:28.879 --> 12:35.879
is negative, then x naught corresponds to
a local maximum. If n is odd then x naught
12:37.939 --> 12:43.209
is a saddle point which means it is neither
a minimum nor a maximum who is call it as
12:43.209 --> 12:48.319
a saddle point, because your first derivative
is 0 which means the slope is still 0 there,
12:48.319 --> 12:55.319
and this is how we make a decision on whether
x is equal to x naught corresponds to a minimum
12:56.189 --> 13:02.600
or a maximum or neither of that.
13:02.600 --> 13:09.600
Let us look at some examples now. So, we started
with the functions with the definitions of
13:10.679 --> 13:17.679
a concave function, convex function and then
we enumerated the necessary, and sufficiency
13:18.649 --> 13:25.649
conditions for local minima of a local minima,
local maxima of a given function and we are
13:25.759 --> 13:32.759
talking about functions of single variables.
Remember we have also not added any conditions
13:33.119 --> 13:40.119
or constraints. So, we are talking about unconstraint
optimization of a single variable or functions
13:40.139 --> 13:45.800
of a single variable.
So, let us look at a function f of x is equal
13:45.800 --> 13:52.129
to 2 x square, if you plot this the function
will look something like this, the function
13:52.129 --> 13:59.129
will look exactly like this not something,
it looks exactly like this. Now, you take
13:59.600 --> 14:05.220
the first derivative d f by d x is equal to
4 x, because we are talking about of a single
14:05.220 --> 14:11.559
variable, I was the full derivative notation
d f by d x this is four x and take the second
14:11.559 --> 14:18.559
derivative d square f by d x square which
is equal to 4 which is positive which means
14:19.569 --> 14:24.600
irrespective of the value of x, the second
derivative is always positive.
14:24.600 --> 14:31.600
So, this is a strictly convex function. So,
this function f of x is equal to 2 x square
14:33.800 --> 14:39.779
is a strictly convex function. One of the
features of the convex function as we saw
14:39.779 --> 14:46.779
in the are previous lecture is that you join
any 2 points on that function, that line will
14:48.189 --> 14:55.189
be always above the function itself. Now,
this we can verify. So, f of alpha x one you
14:59.269 --> 15:06.269
take any 2 points. So, this is x. So, I will
take x 1 here, and x 2 here. You take any
15:07.239 --> 15:14.239
2 points and choose a value of alpha between
0 and 1 and this condition must be satisfied;
15:15.189 --> 15:21.540
that means, f of alpha x 1 plus 1 minus alpha
x 2 which means a function value corresponding
15:21.540 --> 15:28.540
to that particular point must be less than
the point alpha f x 1 plus one minus alpha
15:31.720 --> 15:38.720
f x 2 itself; which means that the straight
line joining 2 points will be above the curve
15:40.220 --> 15:43.329
will be enclosed in the curve in this particular
case.
15:43.329 --> 15:50.149
So, let us seen will choose some x 1 value
x 1 is 0 and x 2 is 2 which means I am taking
15:50.149 --> 15:55.639
x 1 is 0 and x 2 is 2 somewhere here. So,
I am joining those 2 points and I choose alpha
15:55.639 --> 16:02.639
is equal to point 5, just to verified. So,
the LHS which is this part f of alpha x 1
16:03.410 --> 16:10.079
plus 1 minus alpha x 2, f of x is 2 x square.
So, I will use this function at the value
16:10.079 --> 16:17.079
alpha x 1 plus 1 minus alpha x 2. So, I get
f of 1; so, f of 1 is 2 2 into x square that
16:20.809 --> 16:27.809
will be 2, alpha x 1 alpha I am choosing it
point 5 and x 1 is 0 1 minus alpha x 2 x 2
16:29.829 --> 16:36.829
is 2; so, this would be f of 1 and that is
equal to 2. Similarly, value it take alpha
16:37.079 --> 16:44.079
f of x 1 plus 1 minus alpha f of x 2. So,
alpha which is point 5 f of x one which is
16:45.269 --> 16:52.269
0 then 1 minus alpha again point 5 f of x
2 is 2; so, I get four. So, LHS less than
16:53.689 --> 17:00.689
r h s; so, that will be verified that. This
is just a feature of convex functions, remember
17:01.049 --> 17:06.480
this not the definition of convex functions
- the definition of convex functions is here
17:06.480 --> 17:12.500
that is d square f by d x square must be positive
for all values of x in that particular range,
17:12.500 --> 17:15.730
that is a definition.
17:15.730 --> 17:21.800
Let us look at another function. We just take
the mirror image of that will say f of x is
17:21.800 --> 17:28.190
equal to minus 2 x square. So, the function
plots like this. So, if you join any 2 points
17:28.190 --> 17:35.190
here in this curve, this would be below that
curve, this line will be below that curve
17:37.380 --> 17:44.380
and this is what we have seen in the previous
class. So, f of x is equal to minus 2 x square
17:44.890 --> 17:49.310
and therefore, I take the first derivative
that will be minus 4 x and d square f by d
17:49.310 --> 17:54.560
x square would be minus 4 which is always
negative irrespective of the values of x,
17:54.560 --> 18:01.560
the second derivative is always negative.
And therefore, you get the global maximum;
18:01.990 --> 18:06.870
the local maximum also corresponds to the
global maximum. In this case it occurs at
18:06.870 --> 18:13.380
point 0.
So, this is how you determine whether a given
18:13.380 --> 18:20.380
function is a concave function or a convex
function. Now, you can also verify this for
18:24.570 --> 18:31.240
the concave function you have f of alpha x
1 plus 1 minus alpha x 2 must be greater than
18:31.240 --> 18:37.830
alpha f of x 1 plus 1 minus alpha f of x 2,
choose any values of x 1 and x 2 convenient
18:37.830 --> 18:44.830
values of x 1 and x 2 choose a value of alpha
between 0, and 1 and then verify this. I I
18:45.040 --> 18:49.430
want do that, because I just demonstrate that
for the convex functions, similar way you
18:49.430 --> 18:56.430
can do it for concave function. Now, we will
see for given functions how we identify the
19:01.860 --> 19:07.650
stationary points and then see whether the
stationary point corresponds to a minimum
19:07.650 --> 19:12.540
value or a maximum value and so on.
19:12.540 --> 19:19.080
So, we take of function f of x is equal to
3 x cube minus 6 x square plus four x minus
19:19.080 --> 19:26.080
7. The first condition for this to have optimum
values or either minimum value or maximum
19:31.170 --> 19:38.170
value at a given point is that the first derivative
of the function must be equal to 0. So, we
19:42.170 --> 19:49.170
take the first derivative f dash of x and
that will be 9 x square minus 12 x plus 4
19:51.280 --> 19:57.520
and this must be equal to 0. So, f dash x
equal to 0. So, 9 x square minus 12 x plus
19:57.520 --> 20:04.520
4 is equal to 0. You solve for x using this
and that we call it as x naught. So, the solution
20:06.650 --> 20:13.650
for f dash of x is equal to 0 we call it as
x naught, and this we obtain it as 2 by 3.
20:14.670 --> 20:18.960
Now, in this particular case all though we
had quadratic quadratic equation, we get only
20:18.960 --> 20:25.580
one solution, but in general you may get 2
solutions in that situations; if it is the
20:25.580 --> 20:32.290
polynomial of order 3 you may get 3 solutions
or less than 2 of them may be common and so
20:32.290 --> 20:35.740
on.
So, you solve f dash of x is equal to 0 obtain
20:35.740 --> 20:42.740
as many solutions as it yields, and then corresponding
to each of these solutions of a x naught;
20:44.870 --> 20:50.710
you determine f double dash x that is the
second order derivative. So, what it did we
20:50.710 --> 20:57.710
do, we took the first order derivative equated
0, solve for that equation get the stationary
20:57.990 --> 21:04.270
points. In this particular case you obtained
only one stationary point, but there are in
21:04.270 --> 21:10.350
more generally you will get several such stationary
points; that means, x naught is equal to 2
21:10.350 --> 21:13.900
by 3 may be 1 of this solution, x naught is
equal to 0 may be another solution x naught
21:13.900 --> 21:18.160
is equal to minus 1 may be another solution
and so on. So, you may get several stationary
21:18.160 --> 21:23.180
points at each of the stationary points you
have to examine whether the function corresponds
21:23.180 --> 21:30.180
to a minimum or a maximum or neither of them.
So, in this case we will examine at x naught
21:31.490 --> 21:38.260
is equal to 2 by 3 whether the function corresponds
to a minimum or a maximum or neither of them;
21:38.260 --> 21:44.290
how do we do this we go to the second order
derivative and then evaluate the second order
21:44.290 --> 21:48.480
derivative at x naught is equal to 2 by 3.
21:48.480 --> 21:54.840
So, f dash of x is 9 x square minus 12 x plus
4, we take the second order derivative which
21:54.840 --> 22:00.100
means we differentiate this f dash of x again
with respect to x therefore, f double dash
22:00.100 --> 22:07.100
of x will be 18 x minus 12. And this second
order derivative we evaluate at x is equal
22:09.060 --> 22:15.680
to x naught and x naught is 2 by 3. So, f
double dash of x we evaluate at x is equal
22:15.680 --> 22:22.680
to 2 by 3 and these terms have to be equal
to 0. As I said you take the first order derivative
22:27.870 --> 22:33.410
equated to 0 obtain the stationary point go
to the next derivative next order derivative
22:33.410 --> 22:39.360
which is second order derivative and evaluate
the second order derivative at the stationary
22:39.360 --> 22:44.030
point which is 2 by 3 a which is x is equal
to 2 by 3 in our case, and that terms out
22:44.030 --> 22:46.910
to be 0.
If the second order derivative terms out to
22:46.910 --> 22:51.760
be 0, go to the next order derivative which
is the third order derivative and in that
22:51.760 --> 22:58.100
case the third order derivative terms out
to be 18 which is not equal to 0. Once you
22:58.100 --> 23:04.440
obtain the first non-zero derivative, you
look at the order of the derivative, in this
23:04.440 --> 23:09.390
particular case the order of the derivative
is 3 and because the order of the derivative
23:09.390 --> 23:16.390
is odd, the point x is equal to 2 by 3 which
is a stationary point neither corresponds
23:16.700 --> 23:23.500
to a maximum nor a minimum. So, that is the
conclusion here; as n is equal to 3 is odd
23:23.500 --> 23:30.500
n being that order of derivative for which
you are getting a non-zero derivative value,
23:34.470 --> 23:39.860
the function f of x is neither a minimum nor
a maximum at x naught is equal to 2 by 3.
23:39.860 --> 23:43.400
So, this is the conclusion that you derived.
23:43.400 --> 23:50.400
Let us look at another example, where we look
at a function 12 x to the power 5 minus 15
23:51.820 --> 23:58.820
x to the power 4 minus 40 x to the power 3
plus 18 81. We take the first order derivative
24:01.310 --> 24:08.310
f dash of x and equated to 0. So, 16 x to
the power 4 minus 60 x cube minus 120 x square
24:08.610 --> 24:15.610
equated to 0. And you get such a equation;
so, 60 x square you take out x plus 1 into
24:16.460 --> 24:23.270
x minus 2 is equal to 0. So, you get solutions
x is equal to 0, x is equal to minus 1 and
24:23.270 --> 24:30.270
x is equal to 2. So, you got 3 stationary
points here, all of which satisfy the equation
24:31.150 --> 24:38.150
f dash of x is equal to 0. We examine for
local minimum or local maximum or existence
24:41.260 --> 24:48.260
of neither of them at each of these stationary
points. So, you got 3 stationary points corresponding
24:49.640 --> 24:55.820
to each of the stationary 3 stationary points
we examine whether the function has a local
24:55.820 --> 25:02.280
minimum or local maximum or neither of them.
So, we start with x is equal to 0.
25:02.280 --> 25:08.890
So, you got f dash of x from that you get
the second order derivative f double dash
25:08.890 --> 25:15.890
of x. So, f double dash of x in this case
will be 240 x cube minus 180 x square minus
25:15.950 --> 25:22.950
240 x. This second order derivative you evaluate
at x is equal to x naught; let us say x is
25:26.040 --> 25:33.040
equal to 0. So, at x is equal to 0, because
of this nature it terms out to be 0 itself.
25:36.050 --> 25:40.580
Because the second order derivative terms
out to be 0; you go to the third order derivative
25:40.580 --> 25:46.220
still evaluated at x is equal to x naught.
So, third order derivative at x is equal to
25:46.220 --> 25:53.220
x naught terms out to be 240 which is non-zero,
and order is 3 which is odd and therefore,
25:55.360 --> 26:01.050
the point x is equal to 0 corresponds neither
to a minimum nor to a maximum, because the
26:01.050 --> 26:08.050
order is odd here. So, that we exhausted for
x is equal to 0.
26:08.330 --> 26:13.880
Then we take x is equal to minus one, remember
we got 3 solutions here x is equal to 0, x
26:13.880 --> 26:20.880
is equal to minus 1, and x is equal to 2.
We exhausted x is equal to 0 then we go to
26:21.010 --> 26:26.900
x is equal to minus 1. So, at x is equal to
minus 1 we evaluate f double dash of x and
26:26.900 --> 26:33.900
this terms out to be minus 180. So, f double
dash of x evaluated at x is equal to minus
26:35.200 --> 26:42.200
1 is negative and therefore, it corresponds
to a maximum value. So, maximum value occurs
26:44.030 --> 26:51.030
at x is equal to minus 1; once we determine
this then you also evaluate the maximum value
26:52.340 --> 26:57.250
of the function by putting x is equal to minus
1. So, in the f of x which is the original
26:57.250 --> 27:04.250
expression for the function, you put minus
1 and you get f of x is equal to 94. So, the
27:05.360 --> 27:12.360
maximum value of f of x which is a local maximum
of f of x occurring at x is equal to minus
27:12.740 --> 27:19.740
1 is 94. So, we obtained x is equal to 0,
x is equal to minus 1, we we exhausted x is
27:20.830 --> 27:23.180
equal to 0 and x is equal to minus 1.
27:23.180 --> 27:30.150
Next we go to x is equal to 2; determine the
second order derivative evaluated at x is
27:30.150 --> 27:35.760
equal to 2 and see what happens at x is equal
to 2. So, f double dash of x we are determined
27:35.760 --> 27:42.760
as this expression, we substitute x is equal
to 2 which is the third stationary point and
27:43.260 --> 27:50.260
then obtained f double dash of x is equal
to 720 which is the positive value. Because
27:51.440 --> 27:56.840
the order is even which is the second order
derivative, and it terms out that the magnitude
27:56.840 --> 28:02.970
is positive and therefore, the value x is
equal to x naught which is x is equal to 2
28:02.970 --> 28:09.740
in this particular case corresponds to a minimum
value. So, therefore, minimum occurs at x
28:09.740 --> 28:16.250
is equal to 2, and we determine the actual
minimum value as f of x is equal to 12 x to
28:16.250 --> 28:23.250
the power 5 minus 15, we substitute x is equal
to 2 in the original value of function and
28:23.720 --> 28:30.720
that is the minimum value that you get which
is minus 95.
28:31.220 --> 28:38.220
And if you plot the function, the function
plot looks like this. So, at x is equal to
28:41.710 --> 28:47.760
0 you obtained the saddle point. So, the slope
is 0, but it neither corresponds to a local
28:47.760 --> 28:54.130
minimum nor a local maximum. So, this is a
saddle point at x is equal to 0; at x is equal
28:54.130 --> 29:01.130
to minus 1 there is the slope of 0 and it
corresponds to a maximum value, because your
29:01.360 --> 29:08.360
second derivative was negative. At x is equal
to 2, you get the slope as 0 and you get a
29:10.030 --> 29:17.030
minimum value, because the second derivative
was positive. Remember these are all local
29:17.030 --> 29:23.680
these are here you get a local maximum, local
minimum and this is the saddle point. So,
29:23.680 --> 29:30.680
all are at all of these places the slope is
0 and therefore, we are able to obtain the
29:31.510 --> 29:38.140
local minimum and local maximum in this locations.
29:38.140 --> 29:45.140
So, we just completed functions of single
variables first quickly recapsulate what you
29:45.690 --> 29:50.350
did power functions of single variables, we
know how to identify whether the function
29:50.350 --> 29:57.020
is a convex function or concave function;
convex function has a global minimum and therefore,
29:57.020 --> 30:03.830
f double dash of x is positive there; concave
function has a global maximum and therefore,
30:03.830 --> 30:09.780
f double dash of x negative term. So, for
a function of a single variable if the second
30:09.780 --> 30:16.070
order derivative is positive for all values
of x in the particular range, then it corresponds
30:16.070 --> 30:23.070
to a convex function, because you are talking
about a minimum value.
30:23.140 --> 30:28.630
Then we went on to state the necessary conditions
for a function to have a local minimum at
30:28.630 --> 30:33.640
point x is equal to x naught, the necessary
condition is that at that particular point
30:33.640 --> 30:38.980
the slope must be 0 which means f f dash of
x must be equal to 0. So, typically what we
30:38.980 --> 30:44.500
do is given a function you take the first
order derivative equated to 0 solve for a
30:44.500 --> 30:51.500
x is equal to x naught, you may get several
solutions at each of the is each of this solution
30:52.290 --> 30:58.600
which are called as stationary point, you
examine whether the point corresponds to a
30:58.600 --> 31:03.140
local minimum or a local maximum by looking
at the second order derivative, third order
31:03.140 --> 31:10.130
derivative and so on. And we we are now in
position to say whether the stationary point
31:10.130 --> 31:15.610
corresponds to either local minimum or a local
maximum or neither of them in which case it
31:15.610 --> 31:21.840
will be a saddle point.
Now, we will generalize this procedure for
31:21.840 --> 31:28.840
functions of multiple variables. What we did
in the case of single variable, the same principal
31:29.270 --> 31:34.260
we apply to functions of multiple variables
also. Let us say that you have a function
31:34.260 --> 31:41.260
of n variables f of x where x is the vector
of x 1, x 2, x 3, etcetera n variable x n.
31:43.850 --> 31:49.880
The necessary condition is just for excitation
of what we did for the single variable case
31:49.880 --> 31:56.780
that is the first order derivative of the
function with respect to each of the variables
31:56.780 --> 32:03.780
x 1, x 2, x 3, etcetera, x n must be equal
to 0. So, we say d f by d x 1 why use the
32:04.470 --> 32:09.480
partial derivative, because there are n number
of variables d f by d x 1 is equal to d f
32:09.480 --> 32:13.580
by d x 2 is equal to etcetera d f by d x n
with respect to each of the variables the
32:13.580 --> 32:17.150
first order derivative with respect to each
of the variables must be equal to 0, there
32:17.150 --> 32:22.280
is an necessary condition.
So, let us write it more formally round. So,
32:22.280 --> 32:29.280
you have functions of multiple variables and
we denote it has f of x we use capital x to
32:30.330 --> 32:35.970
denote that, this is the vector f of x is
a function of n variables represented by vector
32:35.970 --> 32:42.970
x; x 1 is equal to x 1, x 2, x 3, etcetera,
x n. So, the necessary condition for stationary
32:45.410 --> 32:51.400
point x is equal to x naught is each first
order partial derivative of f of x should
32:51.400 --> 32:58.400
be 0. So, d f by d x 1 is equal to d f by
d x 2 etcetera, d f by d x n is equal to 0.
33:05.170 --> 33:12.170
So, given f of x you obtain the first order
partial differentials with respect to each
33:12.760 --> 33:19.000
of the variables. So, you get n number of
equations, solve for the n number of equations,
33:19.000 --> 33:24.050
get the stationary point x is equal to x star
or x is equal to x naught capital x is equal
33:24.050 --> 33:30.350
to x naught. This is the stationary point
between what you get x 1 naught, x 2 x 2 naught,
33:30.350 --> 33:37.350
etcetera x n naught, as the stationary points.
Then we go to the sufficiency conditions;
33:38.430 --> 33:45.420
what we do in the case of single variable,
we went to the second order derivative. Let
33:45.420 --> 33:51.650
us say you have function of 2 variables; x
1 and x 2 - just look at 2 variables, x 1
33:51.650 --> 33:58.370
and x 2. How many second order derivatives
would be there, you will have 3 second order
33:58.370 --> 34:05.370
derivative d square f by d x 1 square, d square
f by d x 2 square and d square f by d x 1
34:05.670 --> 34:12.670
d x 2. So, you will have 3 second order derivatives.
So, unlike in the first in the case of single
34:14.990 --> 34:21.990
variable, you will have many second order
derivatives in the case of multiple variables;
34:22.169 --> 34:26.210
in case of single variable you had exactly
one second order derivative and therefore,
34:26.210 --> 34:30.020
you variable to make a decision based on the
second order and higher order derivatives
34:30.020 --> 34:36.790
whereas, the number of variables increases
the higher order differential derivatives
34:36.790 --> 34:43.790
will be more than 2 and therefore, the therefore,
you need to construct matrices of second order
34:45.669 --> 34:50.889
derivatives to decide whether the stationary
point that you so, obtain x is equal to x
34:50.889 --> 34:57.580
naught corresponds to a minimum or a maximum
or a neither of them. So, we formulate what
34:57.580 --> 35:04.580
is called as the hessian matrix. The hessian
matrix is a matrix of the second order derivatives,
35:05.230 --> 35:09.480
and we make our decisions based on the hessian
matrix.
35:09.480 --> 35:16.480
How do we formulate the hessian matrix, the
h of a given function h of f of x in the particular
35:20.970 --> 35:27.970
case is a hessian matrix of function f of
x; we define the hessian matrix as the matrix
35:29.710 --> 35:36.670
which is the n by n size, n being the number
of variables in this case of second order
35:36.670 --> 35:42.720
derivatives. So, we are write x 1, x 2, etcetera,
x n here; x 1, x 2, etcetera x n, here. Take
35:42.720 --> 35:49.720
the second order derivative d square f of
x by d x 1 square, there is x one with respect
35:50.080 --> 35:57.080
to x 1, x 1 with respect to x 2 which is d
square f by d x 1, d x 2 then x 1 and x n
35:59.740 --> 36:06.740
d square f by d x 1 d x n like this for example,
here you get d square f by d x 2 x 1; the
36:09.380 --> 36:16.380
d square f by x 2 x 2 that is the x 2 square
x 2 x n like this you formulate the second
36:16.800 --> 36:23.800
order derivatives; for x n it will be d square
f by d x 1 d x 2 and so on. So, you get a
36:24.240 --> 36:31.240
n by n matrix of the second order derivatives.
Now, this is the hessian matrix. The hessian
36:32.190 --> 36:38.610
matrix has to be evaluated at x is equal to
x naught. So, this hessian matrix which so
36:38.610 --> 36:43.300
formulate, we evaluate this hessian matrix
at x is equal to x naught; what is the x is
36:43.300 --> 36:47.430
equal to x naught? That is the stationary
point that we obtain here. So, you obtain
36:47.430 --> 36:52.910
the stationary point at x is equal to x naught,
formulate the hessian matrix evaluate the
36:52.910 --> 36:58.730
hessian matrix at x is equal to x naught,
and then make your decisions of whether the
36:58.730 --> 37:05.730
point x is equal to x naught, corresponds
to a global a local minimum or a local maximum;
37:06.119 --> 37:10.020
why we will do that now.
37:10.020 --> 37:17.020
So, the sufficiency condition is if the hessian
matrix that you show determine just now and
37:19.100 --> 37:26.100
evaluated at x is equal to x naught, if the
hessian matrix is positive definite; there
37:30.850 --> 37:37.850
is a mistake here. I will just convert this,
if the hessian matrix is negative definite
37:38.640 --> 37:45.640
at x is equal to x naught then it corresponds
to a minimum value. And if it is negative
37:45.680 --> 37:52.680
definite at x is equal x naught then it corresponds
to a maximum point. So, I repeat you formulate
37:54.970 --> 38:01.970
the hessian matrix which is the n by n matrix
of the second order derivatives. Evaluate
38:03.030 --> 38:09.230
the hessian matrix at x is equal to x naught.
If the hessian matrix is positive definite
38:09.230 --> 38:15.990
then the point corresponds to a minimum and
if it is the negative definite it corresponds
38:15.990 --> 38:22.380
to a maximum.
Now, I use the terms positive definite and
38:22.380 --> 38:28.800
negative definite; how do we determine whether
a given matrix is positive definite and now
38:28.800 --> 38:35.800
or or it is the negative definite; of we define
the matrix to be positive or positive definite
38:36.860 --> 38:43.860
or negative definite only for a square matrices.
So, if a square matrix has its eigen values
38:45.990 --> 38:52.180
all of which are positive, then it is called
as the positive definite matrix; that means,
38:52.180 --> 38:58.640
if all the eigen values are positive then
the square matrix is a positive definite matrix.
38:58.640 --> 39:05.640
If all the eigen values are negative then
it is the negative definite matrix. If some
39:06.240 --> 39:11.250
eigen values are positive and some are negative
then it is the neither of positive definite
39:11.250 --> 39:18.250
nor a negative definite.
So, we make decisions at x is equal to x naught
39:21.220 --> 39:28.220
based on whether the h matrix or the hessian
matrix evaluated at x is equal to x naught
39:29.440 --> 39:34.890
is in fact, of positive definite matrix or
a negative definite matrix. If it is neither
39:34.890 --> 39:40.300
of them then, the point x is equal to x naught
which is the stationary point corresponds
39:40.300 --> 39:46.570
to neither a minimum nor a maximum.
39:46.570 --> 39:53.570
And then I I said that the hessian matrix
is positive definite, if all the eigen values
39:53.850 --> 39:59.950
are positive. So, we must know how to determine
the eigen values; I hope all of you know how
39:59.950 --> 40:06.630
to determine, this is the simple characteristic
equation lambda I minus H, H is the a symmetric
40:06.630 --> 40:11.830
determinant of that you set it to the 0 and
then solve for lambdas. So, the lambdas are
40:11.830 --> 40:18.830
though eigen values when we solve the numerical
examples it will become.
40:19.609 --> 40:26.310
So, in the case of multiple variable, in the
case of functions of multiple variables we
40:26.310 --> 40:31.910
get the first order derivatives equated to
0, handle if you have a n variables you have
40:31.910 --> 40:36.869
n first order derivatives and therefore, you
have n equations solve for the n equations
40:36.869 --> 40:43.430
you get the stationary point, capital x is
equal to capital x naught at the stationary
40:43.430 --> 40:49.260
point you evaluate the hessian matrix - hessian
matrix is the n by n matrix of the second
40:49.260 --> 40:54.130
order derivatives.
You evaluate the hessian matrix at the stationary
40:54.130 --> 41:00.270
point x is equal to x naught examine whether
at x is equal to x naught, the hessian matrix
41:00.270 --> 41:05.460
is positive definite or negative definite.
If the hessian matrix is positive definite
41:05.460 --> 41:10.550
the point x is equal to x naught corresponds
to a minimum; if the hessian matrix is negative
41:10.550 --> 41:17.550
definite the x is equal to x naught corresponds
to a maximum. To determine whether the hessian
41:17.550 --> 41:23.170
matrix is positive definite or negative definite
you use the eigen values; if all the eigen
41:23.170 --> 41:30.170
values are negative, then the hessian
matrix is negative definite matrix, if some
of them some of the eigen values are positive,
41:38.850 --> 41:43.850
some of them are negative then the hessian
matrix is neither positive definite nor negative
41:43.850 --> 41:48.100
definite; in which case the stationary point
x is equal to x naught neither corresponds
41:48.100 --> 41:53.140
to a maximum value nor corresponds to a minimum
value. So, this is what we do in the case
41:53.140 --> 41:58.140
of multiple functions of multiple variables.
41:58.140 --> 42:05.140
Now, similar to what we did in the case of
functions of single variables; let us say
42:05.700 --> 42:11.590
that the hessian matrix which is the matrix
of the second order derivatives is positive
42:11.590 --> 42:18.200
definite irrespective of a values of x at
are using which means the entire range of
42:18.200 --> 42:23.320
the entire range of capital x over which the
functions has been defined, the hessian matrix
42:23.320 --> 42:30.320
is positive always positive. Then the function
is strictly convex. So, the multiple function
42:32.790 --> 42:38.010
of multiple variables is strictly convex which
means the hessian matrix being positive, irrespective
42:38.010 --> 42:45.010
value of a values of the variables x, then
the function is strictly convex in which case
42:45.869 --> 42:52.869
the local minimum also corresponds to the
global minimum. If all the eigen values are
42:54.119 --> 43:01.119
negative, then the functions corresponds to
function is strictly concave which means the
43:01.470 --> 43:08.470
local maximum is also is local global maximum;
if some over eigen values are positive and
43:08.480 --> 43:15.480
some are negative as I said it neither corresponds
to it is neither convex nor concave in that
43:16.160 --> 43:21.160
particular range.
43:21.160 --> 43:27.810
We will just summarized this whatever I just
said whether the function is minimum or maximum
43:27.810 --> 43:34.810
at x is equal to x naught depends on nature
of eigen values of its hessian matrix evaluated
43:35.130 --> 43:42.130
at x is equal x naught; that is the hessian
matrix is evaluated at x is equal to x naught.
43:44.130 --> 43:49.520
If all eigen values are positive at x naught,
x naught is a local minimum this is what we
43:49.520 --> 43:56.520
solve. If all eigen values are positive for
all possible values of x then x naught is
43:57.760 --> 44:04.520
a global minimum, because that is corresponds
to a convex function. So, if if the function
44:04.520 --> 44:11.520
is convex then the local minimum, there is
a local minimum also corresponds to the global
44:11.950 --> 44:18.859
minimum. So, that is what you mean here. If
all eigen eigen values are negative at x naught,
44:18.859 --> 44:24.310
x naught is the local maximum; then if all
eigen values are negative for all possible
44:24.310 --> 44:31.000
values of x, then x naught is a global maximum.
Similarly, if some of them are positive, some
44:31.000 --> 44:35.869
of them are negative where you are able to
make decision on whether it is a local minimum
44:35.869 --> 44:41.650
or I am sorry you are able to stay that the
point x is equal to x naught corresponds neither
44:41.650 --> 44:48.400
to a local minimum nor to a local maximum.
44:48.400 --> 44:55.400
Let us look an example of multiple variables.
We take 2 variables now. So, f of x is equal
44:59.650 --> 45:06.650
to x 1 square plus x 2 square minus 4 x 1
minus 2 x square 2 x 2 plus 5. So, the first
45:12.710 --> 45:19.420
step in optimizing in obtaining the optimal
values of functions of multiple variables
45:19.420 --> 45:25.990
is to take the first derivative with respect
to each of the variables; you have 2 variables
45:25.990 --> 45:30.480
x 1 and x 2 corresponding to each of the 2
variables, we take the first derivatives.
45:30.480 --> 45:37.480
So, d f by d x 1, I use the notation for partial
derivatives d f by d x 1 is equal to 2 x 1
45:38.660 --> 45:45.660
2 x 1 minus 4, I am differentiating this function
with respect to x 1, and set to 0 2 x 1 minus
45:47.300 --> 45:51.130
4 is equal to 0.
Similarly, I differentiate this function with
45:51.130 --> 45:58.130
respect to x 2 that will be 2 x 2 minus 2
is equal to 0, and by solving this I get x
45:58.359 --> 46:05.359
1 is equal to 2 and x 2 is equal to 1. So,
the stationary point I obtained it as (2,1)
46:05.920 --> 46:12.920
that is the x 1 is equal to 2 and x 2 is equal
to 1, this is the stationary point. Now, corresponds
46:13.340 --> 46:20.340
to this the stationary point I will now evaluate
the hessian matrix for the function. So, formulate
46:21.850 --> 46:28.850
the hessian matrix which will be a 2 by 2
matrix of the second order derivatives.
46:28.920 --> 46:35.619
Let us look at the hessian matrix. There is
a x 1 here, x 2 here, x 1 here, x 2 here.
46:35.619 --> 46:42.619
So, we have d square f by d x 1 square, d
square f by d x 1 d x 2, d square f by d x
46:43.280 --> 46:50.280
2 d x 1 and d square f by d x 2 square; these
2 will be the same derivatives. This has to
46:53.680 --> 47:00.590
be evaluated at (2,1) which is the stationary
point that you have obtained. So, we formulate
47:00.590 --> 47:05.880
the hessian matrix and evaluate it at (2,1).
47:05.880 --> 47:12.880
So, f of x is this d f by d x 1 is 2 x 1 minus
4 and therefore, d square f by d x 1 square
47:17.840 --> 47:24.480
is we are differentiating this with respect
to x 1 and therefore, we get d square f by
47:24.480 --> 47:31.480
d x 1 square is equal to 2, d square f by
d x 1 d x 2 you differentiate this with respect
47:32.560 --> 47:39.560
to x 2 that till 0, and you have got d f by
d x 2 as 2 x 2 minus 2 and d square f by d
47:42.660 --> 47:49.660
x 2 d x 1 will be 0 similar to this d square
f by d x 2 square will be equal to 2. What
47:54.960 --> 48:01.960
happens in this case is the irrespective of
the values of x 1 you are second order derivative
48:02.130 --> 48:07.700
is 2 here; irrespective of the values of x
2 your second derivative is 2 here; and irrespective
48:07.700 --> 48:13.570
of the values of x one and x 2 your second
order derivatives with respect to x 1 and
48:13.570 --> 48:20.570
x 2 or 0 here. So, we will examine whether
this is the positive definite matrix or an
48:20.790 --> 48:27.790
negative definite matrix; first and see whether
this remains irrespective of the values of
48:29.500 --> 48:36.500
x 1 and x 2. So, the hessian matrix is 2 0
0 2 these are the values, d square f by d
48:39.050 --> 48:45.140
x one square and d square f by d x 2 square
and d square f by d x 1 d x 2.
48:45.140 --> 48:52.140
So, the hessian matrix is 2 0 0 2, let us
determine the eigen values of this which means
48:52.690 --> 48:59.690
I take the determinant lambda I minus H is
equal to 0. So, H of f of X, but we can typically
49:02.359 --> 49:09.359
write it as H itself. So, we will write lambda
I minus H which will be lambda minus 2 0 0
49:11.720 --> 49:18.720
lambda minus 2; that is what we get here.
So, you get lambda minus 2 the whole square
49:18.859 --> 49:25.859
is equal to 0 this is the determinant, here
this is determinant and this we are setting
49:29.080 --> 49:35.740
it as 0. So, lambda minus two the whole square
is equal to 0. So, we get the solutions for
49:35.740 --> 49:42.740
lambda as lambda 1 is equal to 0 and lambda
2 is equal to 2.
49:45.260 --> 49:51.050
Which means the both the eigen values that
you obtained or in facts same lambda is equal
49:51.050 --> 49:58.050
to 2 if what you got both the solutions are
positive and therefore, the H matrix is positive
49:58.530 --> 50:05.530
definite. And therefore, the solution that
we obtained that is the stationary point that
50:06.000 --> 50:13.000
we obtained namely x is equal to (2,1). In
fact, corresponds to a local minimum further
50:14.880 --> 50:21.880
why we set local minimum, it is because the
H matrix is positive definite, because it
50:22.760 --> 50:26.730
is positive definite it corresponds to a minimum
value and therefore, the stationary point
50:26.730 --> 50:33.730
that we just obtained namely (2,1) is in fact,
a local minimum. Further, because the h matrix
50:35.280 --> 50:42.280
is positive definite irrespective of the values
of x, remember here we got this matrix (2,0),
50:43.099 --> 50:50.099
(0,2) irrespective of the values of x that
we substitute.
50:50.460 --> 50:55.090
If this where a function of x 1 and x 2 then
you could have substituted at x 1 is equal
50:55.090 --> 51:02.090
to 2, x 1 x 2 is equal to 1 and then obtained
the h matrix evaluated at that point, but
51:03.130 --> 51:08.940
because this is independent of x 1 and x 2
and we have a obtain this matrix to be positive
51:08.940 --> 51:14.710
definite, it means that H remains to be positive
definite irrespective of the values of x and
51:14.710 --> 51:21.010
therefore, the function is a convex function.
Because the function is concave function the
51:21.010 --> 51:27.369
local minimum that you obtained at this particular
point x is equal to (2,1) is also a global
51:27.369 --> 51:30.950
minimum.
So, as the hessian matrix does not depend
51:30.950 --> 51:36.460
on x 1 and x 2, and it is positive definite
matrix the function is strictly convex and
51:36.460 --> 51:43.460
therefore, the local minimum is also the global
minimum. Let us look at I hope this problem
51:45.330 --> 51:50.010
is understood correctly what was essentially
did was that we obtain the stationary point
51:50.010 --> 51:54.930
by taking the first order derivatives, there
were 2 variables in the case and therefore,
51:54.930 --> 52:01.930
you get two equations when we take the first
order derivative, solve for the the variables
52:03.810 --> 52:09.150
x 1 and x 2; this consecutive stationary point
- at the stationary point you evaluate the
52:09.150 --> 52:13.510
hessian matrix - the hessian matrix in this
particular case will be a square matrix of
52:13.510 --> 52:20.510
order two; this hessian matrix has to be evaluated
at the stationary point, and then examine
52:21.119 --> 52:27.770
if this is the hessian matrix if the hessian
matrix is positive definite or negative definite.
52:27.770 --> 52:32.270
If a hessian matrix is positive definite,
then the stationary point corresponds to a
52:32.270 --> 52:36.540
minimum value; if the hessian matrix is negative
definite, then the stationary point corresponds
52:36.540 --> 52:43.540
to a maximum value. In the in the particular
example that we examine the hessian matrix
52:43.810 --> 52:48.840
is remained positive definite irrespective
of the values of x 1 and x 2; which means
52:48.840 --> 52:53.780
all the values of x 1 and x 2 hessian matrix
remains positive definite. And therefore,
52:53.780 --> 52:59.060
the function corresponds to a a function is
a convex function and therefore, the local
52:59.060 --> 53:06.060
minimum that you obtain at the stationary
point is in fact, the global minimum.
53:06.730 --> 53:13.690
The same thing is valid for concave function
also where the stationary point corresponds
53:13.690 --> 53:19.820
to a local maximum and therefore, the local
maximum also corresponds to a global maximum,
53:19.820 --> 53:26.820
if the function is a concave function. We
will see some are examples where we may get
53:29.619 --> 53:34.349
a concave function or we may get neither a
concave nor a convex function in the entire
53:34.349 --> 53:41.349
range and therefore, we may get some stationary
points corresponding to a local minimum, some
53:42.609 --> 53:46.609
corresponds to corresponding to a local maximum
and so on.
53:46.609 --> 53:53.609
That we will examine throws some other examples
in the next lecture. So, to summarize in today
53:55.200 --> 54:02.200
lecture, essentially we have started with
functions of single variables and seen the
54:02.770 --> 54:09.770
necessary and the sufficiency conditions for
functions of single variables to have a local
54:10.490 --> 54:17.490
minimum or a local maximum at a given point.
So, what we do is we take the first order
54:18.640 --> 54:25.500
differential, first derivative we take and
equated to 0, solve for that and you get the
54:25.500 --> 54:29.530
stationary point.
Now, at the stationary point the examine the
54:29.530 --> 54:34.270
higher order derivatives, second order derivative,
if the second order derivative is negative
54:34.270 --> 54:39.070
then the stationary point x is equal to x
naught corresponds to a maximum; if the second
54:39.070 --> 54:46.070
order derivative is positive then it corresponds
to a minimum; or I again repeat always associate
54:47.460 --> 54:54.460
positive values with minimum, that is positive
values of the second order derivative or the
54:54.660 --> 54:58.780
higher order derivative where we are able
to make a decision positive always to corresponds
54:58.780 --> 55:04.089
to a minimum, negative always corresponds
to a maximum.
55:04.089 --> 55:09.580
And if it is if the second order derivative
was also 0 then you go to higher order derivative,
55:09.580 --> 55:16.580
third order, fourth order, fifth order, etcetera,
and get the first derivative which is non-zero,
55:18.190 --> 55:23.190
then look at the order. If the order is odd
then this stationary point is neither a minimum
55:23.190 --> 55:30.190
nor a maximum. If the order is even then you
then you check the magnitude of that particular
55:30.290 --> 55:34.060
derivative, if the magnitude is positive it
corresponds to a minimum, if the magnitude
55:34.060 --> 55:41.060
is negative it corresponds to a maximum. We
saw some examples, where we got a combination
55:43.020 --> 55:47.359
of these cases where you can you can identify
a local minimum or local maximum and we are
55:47.359 --> 55:53.000
also able to stay that the point corresponds
to neither a minimum nor a maximum.
55:53.000 --> 55:58.450
Then we went on to examine functions of multiple
variables where essentially the same principle
55:58.450 --> 56:03.859
wholes you first take the first order derivative
equal to 0, if there are n number of variables
56:03.859 --> 56:08.990
you get n number of equations, solve for this
n variables using these n number of variables
56:08.990 --> 56:14.080
that defines the stationary point, capital
x is equal to x naught where x is the vector
56:14.080 --> 56:21.080
of n variables then you formulate the hessian
matrix hessian, matrix is the n by n matrix
56:21.290 --> 56:27.970
n by n square matrix of second order derivatives,
evaluate the hessian matrix at x equal to
56:27.970 --> 56:33.220
x naught which is the stationary point; if
at x equal to x naught your hessian matrix
56:33.220 --> 56:38.810
is a positive definite matrix, a point of
x equal to x naught corresponds to a minimum;
56:38.810 --> 56:43.040
if the hessian matrix evaluated at x is equal
to x naught is the negative definite matrix
56:43.040 --> 56:46.540
then the point x is equal to x naught corresponds
to a maximum.
56:46.540 --> 56:53.540
Always associate positive with minimum values
negative with maximum values. When you are
56:54.290 --> 57:01.290
making their decisions based on the derivatives.
Then, we also saw that if the hessian matrix
57:02.640 --> 57:08.540
remains positive definite illustrative of
the values of the x, that will use which means
57:08.540 --> 57:13.700
in the entire range of the definition of the
function, if the hessian matrix remains positive
57:13.700 --> 57:18.560
definite then the function corresponds to
a convex function and therefore, the local
57:18.560 --> 57:24.670
minimum also corresponds to the global minimum;
local minimum is also the global minimum.
57:24.670 --> 57:28.849
Similarly, if the hessian matrix remains the
negative in the entire range of a function
57:28.849 --> 57:35.849
definition then the local maximum corresponds
to the global maximum. So, we will continue
57:36.849 --> 57:43.849
this discussion on multiple variables in the
next class, I will cover another two examples
57:47.320 --> 57:53.680
and that completes the unconstrained optimization.
Remember the class of optimization technique
57:53.680 --> 57:59.640
that we are dealing with or unconstrained
optimization, because we are not putting any
57:59.640 --> 58:05.050
conditions, we are just stating a function
and for that function to have a local minimum
58:05.050 --> 58:10.010
local maximum, global minimum global maximum,
we are setting out the conditions for that;
58:10.010 --> 58:15.980
when we start put constrains on to this, then
the problems becomes much more complicated.
58:15.980 --> 58:21.019
We will see those in the subsequent classes,
thank you for your attention.