All language subtitles for 05_gradient-descent-for-linear-regression.en

Previously, you took a look at

the linear regression model
and then the cost function,

and then the gradient
descent algorithm.

In this video, we're going
to pull out together and use

the squared error
cost function for

the linear regression model
with gradient descent.

This will allow us to train

the linear regression
model to fit

a straight line to achieve

the training data.
Let's get to it.

Here's the linear
regression model.

To the right is the squared
error cost function.

Below is the gradient
descent algorithm.

It turns out if you
calculate these derivatives,

these are the terms
you would get.

The derivative with respect
to W is this 1 over m,

sum of i equals 1 through
m. Then the error term,

that is the difference
between the predicted and

the actual values times
the input feature xi.

The derivative with respect to b

is this formula over here,

which looks the same
as the equation above,

except that it doesn't have
that xi term at the end.

If you use these
formulas to compute

these two derivatives and

implements gradient descent
this way, it will work.

Now, you may be wondering,

where did I get
these formulas from?

They're derived using calculus.

If you want to see
the full derivation,

I'll quickly run through

the derivation on
the next slide.

But if you don't
remember or aren't

interested in the calculus,
don't worry about it.

You can skip the materials on

the next slide entirely
and still be able

to implement gradient
descent and finish

this class and everything
will work just fine.

In this slide, which is one of

the most mathematical slide
of the entire specialization,

and again is
completely optional,

we'll show you how to calculate
the derivative terms.

Let's start with the first term.

The derivative of the
cost function J with

respect to w. We'll
start by plugging in

the definition of the cost
function J. J of WP is this.

1 over 2m times this sum of
the squared error terms.

Now remember also that f of wb

of X^i is equal to
this term over here,

which is WX^i plus b.

What we would like to do
is compute the derivative,

also called the partial
derivative with respect to

w of this equation right
here on the right.

If you taken a
calculus class before,

and again is totally
fine if you haven't,

you may know that by
the rules of calculus,

the derivative is equal
to this term over here.

Which is why the two here
and two here cancel out,

leaving us with this equation

that you saw on the
previous slide.

This is why we had to find
the cost function with the

1.5 earlier this week

is because it makes the
partial derivative neater.

It cancels out the
two that appears

from computing the derivative.

For the other derivative
with respect to b,

this is quite similar.

I can write it out like
this, and once again,

plugging the definition of f

of X^i, giving this equation.

By the rules of calculus,

this is equal to this

where there's no X^i
anymore at the end.

The 2's cancel one small
and you end up with

this expression for the
derivative with respect to b.

Now you have these two
expressions for the derivatives.

You can plug them into the
gradient descent algorithm.

Here's the gradient
descent algorithm

for linear regression.

You repeatedly carry
out these updates

to w and b until convergence.

Remember that this f of x is
a linear regression model,

so as equal to w times x plus b.

This expression here is

the derivative of the cost
function with respect to

w. This expression is

the derivative of the cost
function with respect to b.

Just as a reminder,

you want to update w and b
simultaneously on each step.

Now, let's get familiar with
how gradient descent works.

One the shoe we saw with
gradient descent is that it can

lead to a local minimum
instead of a global minimum.

Whether global minimum
means the point that has

the lowest possible value
for the cost function

J of all possible points.

You may recall this surface
plot that looks like

an outdoor park with
a few hills with

the process and the birds
as a relaxing Hobo Hill.

This function has more
than one local minimum.

Remember, depending on where

you initialize the
parameters w and b,

you can end up at
different local minima.

You can end up here,

or you can end up here.

But it turns out
when you're using

a squared error cost function
with linear regression,

the cost function
does not and will

never have multiple
local minima.

It has a single global minimum

because of this bowl-shape.

The technical term
for this is that

this cost function is
a convex function.

Informally, a convex function

is of bowl-shaped function and

it cannot have any local minima

other than the single
global minimum.

When you implement gradient
descent on a convex function,

one nice property is that so

long as you're learning rate
is chosen appropriately,

it will always converge
to the global minimum.

Congratulations,
you now know how to

implement gradient descent
for linear regression.

We have just one last
video for this week.

That video, we'll see
this algorithm in action.

Let's go to that last video.