Original subtitles

You've learned about gradient descents about

multiple linear regression and also vectorization.

Let's put it all together to

implement gradient descent for

multiple linear regression with

vectorization. This would be cool.

Let's quickly review what

multiple linear regression look like.

Using our previous notation,

let's see how you can write it more

succinctly using vector notation.

We have parameters w_1 to w_n as well as b.

But instead of thinking of

w_1 to w_n as separate numbers,

that is separate parameters,

let's start to collect all of the w's into

a vector w so that now w is

a vector of length n.

We're just going to think of the parameters of

this model as a vector w,

as well as b,

where b is still a number same as before.

Whereas before we had to find

multiple linear regression like this,

now using vector notation,

we can write the model as f_w,

b of x equals the vector

w dot product with the vector x plus b.

Remember that this dot here means.product.

Our cost function can be defined as J

of w_1 through w_n, b.

But instead of just thinking of J as a function of these

and different parameters w_j as well as b,

we're going to write J as a function of

parameter vector w and the number b.

This w_1 through w_n is replaced by this vector W and

J now takes this input of

vector w and a number b and returns a number.

Here's what gradient descent looks like.

We're going to repeatedly update each parameter w_j to

be w_j minus Alpha times the derivative of the cost J,

where J has parameters w_1 through w_n and b.

Once again, we just write this as J

of vector w and number b.

Let's see what this looks like when you implement

gradient descent and in particular,

let's take a look at the derivative term.

We'll see that gradient descent becomes just a little bit

different with multiple features

compared to just one feature.

Here's what we had when we had

gradient descent with one feature.

We had an update rule for w and

a separate update rule for b. Hopefully,

these look familiar to you.

This term here is the derivative of the cost function J

with respect to the parameter w. Similarly,

we have an update rule for parameter b,

with univariate regression, we had only one feature.

We call that feature xi without any subscript.

Now, here's a new notation for where we have n features,

where n is two or more.

We get this update rule for gradient descent.

Update w_1 to be w_1 minus

Alpha times this expression here

and this formula is actually

the derivative of the cost J with respect to w_1.

The formula for the derivative of

J with respect to w_1 on

the right looks very similar to

the case of one feature on the left.

The error term still takes

a prediction f of x minus the target y.

One difference is that w and x are now vectors

and just as w on the left

has now become w_1 here on the right,

xi here on the left is now instead xi _1 here on

the right and this is just for J equals 1.

For multiple linear regression,

we have J ranging from 1 through n and

so we'll update the parameters w_1,

w_2, all the way up to w_n,

and then as before, we'll update b.

If you implement this,

you get gradient descent for multiple regression.

That's it for gradient descent for multiple regression.

Before moving on from this video,

I want to make a quick aside or a quick side note on

an alternative way for finding

w and b for linear regression.

This method is called the normal equation.

Whereas it turns out gradient descent is a great method

for minimizing the cost function J to find w and b,

there is one other algorithm that works only

for linear regression and pretty much none of

the other algorithms you see in this specialization

for solving for w and b

and this other method does not need

an iterative gradient descent algorithm.

Called the normal equation method,

it turns out to be possible to use

an advanced linear algebra library to just solve for

w and b all in one goal without iterations.

Some disadvantages of the normal equation method are;

first unlike gradient descent,

this is not generalized to other learning algorithms,

such as the logistic regression algorithm

that you'll learn about next week

or the neural networks or

other algorithms you see later in this specialization.

The normal equation method is also quite

slow if the number of features and this large.

Almost no machine learning

practitioners should implement the normal equation method

themselves but if you're using

a mature machine learning

library and call linear regression,

there is a chance that on the backend,

it'll be using this to solve for w and b.

If you're ever in the job interview

and hear the term normal equation,

that's what this refers to.

Don't worry about the details of how

the normal equation works.

Just be aware that some machine learning libraries

may use this complicated method

in the back-end to solve for w and b.

But for most learning algorithms,

including how you implement linear regression yourself,

gradient descents offer a better way to get the job done.

In the optional lab that follows this video,

you'll see how to define a multiple regression model

encode and also how to calculate the prediction f of x.

You'll also see how to calculate the cost and

implement gradient descent for

a multiple linear regression model.

This will be using Python's NumPy library.

If any of the code looks very new,

that's okay but you should feel free

also to take a look at the previous optional lab that

introduces NumPy and vectorization for a refresher

of NumPy functions and how to implement those in code.

That's it. You now know multiple linear regression.

This is probably the single most widely used

learning algorithm in the world today. But there's more.

With just a few tricks such

as picking and scaling features

appropriately and also choosing

the learning rate alpha appropriately,

you'd really make this work much better.

Just a few more videos to go for this week.

Let's go on to the next video

to see those little tricks that will

help you make multiple linear

regression work much better.