All language subtitles for 02_addressing-overfitting.en

Later in this specialization,

we'll talk about debugging and

diagnosing things that can go

wrong with learning algorithms.

You'll also learn about
specific tools to

recognize when overfitting and

underfitting may be occurring.

But for now, when you think
overfitting has occurred,

lets talk about what you
can do to address it.

Let's say you fit a model

and it has high
variance, is overfit.

Here's our overfit house
price prediction model.

One way to address
this problem is to

collect more training
data, that's one option.

If you're able to get more data,

that is more training examples

on sizes and prices of houses,

then with the larger
training set,

the learning algorithm
will learn to

fit a function that
is less wiggly.

You can continue to fit

a high order polynomial

or some of the function
with a lot of features,

and if you have enough
training examples,

it will still do okay.

To summarize, the
number one tool you can

use against overfitting is
to get more training data.

Now, getting more data
isn't always an option.

Maybe only so many houses have

been sold in this location,

so maybe there just isn't
more data to be add.

But when the data is available,

this can work really well.

A second option for addressing

overfitting is to see if
you can use fewer features.

In the previous video,

our models features
included the size x,

as well as the size squared,
and this x squared,

and x cubed and x^4 and so on.

These were a lot of
polynomial features.

In that case, one way to
reduce overfitting is to

just not use so many of
these polynomial features.

But now let's look at
a different example.

Maybe you have a lot of
different features of

a house of which to try
to predict its price,

ranging from the size,
number of bedrooms,

number of floors, the age,

average income of
the neighborhood,

and so on and so forth,

total distance to the
nearest coffee shop.

It turns out that if you
have a lot of features like

these but don't have
enough training data,

then your learning algorithm may

also overfit to
your training set.

Now instead of using
all 100 features,

if we were to pick just a
subset of the most useful ones,

maybe size, bedrooms,

and the age of the house.

If you think those are the
most relevant features,

then using just that
smallest subset of features,

you may find that your model
no longer overfits as badly.

Choosing the most appropriate
set of features to

use is sometimes also
called feature selection.

One way you could
do so is to use

your intuition to
choose what you

think is the best
set of features,

what's most relevant for
predicting the price.

Now, one disadvantage
of feature selection

is that by using only a
subset of the features,

the algorithm is
throwing away some of

the information that you
have about the houses.

For example, maybe all
of these features,

all 100 of them are actually

useful for predicting
the price of a house.

Maybe you don't want
to throw away some of

the information by throwing
away some of the features.

Later in Course 2,

you'll also see some
algorithms for automatically

choosing the most
appropriate set of

features to use for
our prediction task.

Now, this takes us to

the third option for
reducing overfitting.

This technique, which we'll
look at in even greater depth

in the next video is
called regularization.

If you look at an overfit model,

here's a model using
polynomial features: x,

x squared, x cubed, and so on.

You find that the parameters
are often relatively large.

Now if you were to

eliminate some of
these features, say,

if you were to eliminate
the feature x4,

that corresponds to setting
this parameter to 0.

So setting a parameter to 0

is equivalent to
eliminating a feature,

which is what we saw
on the previous slide.

It turns out that regularization

is a way to more gently reduce

the impacts of some of
the features without

doing something as harsh as
eliminating it outright.

What regularization
does is encourage

the learning algorithm
to shrink the values of

the parameters
without necessarily

demanding that the parameter
is set to exactly 0.

It turns out that
even if you fit

a higher order
polynomial like this,

so long as you can get
the algorithm to use

smaller parameter values: w1,

w2, w3, w4.

You end up with a curve
that ends up fitting

the training data much better.

So what regularization does,

is it lets you keep
all of your features,

but they just prevents
the features from

having an overly large effect,

which is what sometimes
can cause overfitting.

By the way, by convention,

we normally just reduce the
size of the wj parameters,

that is w1 through wn.

It doesn't make a huge
difference whether you

regularize the
parameter b as well,

you could do so if you
want or not if you don't.

I usually don't and it's just

fine to regularize w1, w2,

all the way to wn,

but not really encourage
b to become smaller.

In practice, it should make
very little difference

whether you also
regularize b or not.

To recap, these are

the three ways you saw in

this video for
addressing overfitting.

One, collect more data.

If you can get more data,

this can really help
reduce overfitting.

Sometimes that's not possible.

In which case, some of
the options are, two,

try selecting and using only
a subset of the features.

You'll learn more about
feature selection in Course 2.

Three would be to

reduce the size of the
parameters using regularization.

This will be the subject
of the next video as well.

Just for myself, I use
regularization all the time.

So this is a very
useful technique

for training
learning algorithms,

including neural
networks specifically,

which you'll see later in
this specialization as well.

I hope you'll also check out

the optional lab on overfitting.

In the lab, you'll be able
to see different examples of

overfitting and
adjust those examples

by clicking on
options in the plots.

You'll also be able to add

your own data points
by clicking on

the plot and see how that
changes the curve that is fit.

You can also try examples
for both regression and

classification and you will

change the degree of
the polynomial to be x,

x squared, x cubed, and so on.

The lab also lets you play with

two different options for
addressing overfitting.

You can add additional
training data to

reduce overfitting and
you can also select which

features to include
or to exclude

as another way to try
to reduce overfitting.

Please take a look at a lab,

which I hope will help you
build your intuition about

overfitting as well as some
methods for addressing it.

In this video, you
also saw the idea of

regularization at a
relatively high level.

I realize that all
of these details on

regularization may not fully
make sense to you yet.

But in the next video,

we'll start to formulate
exactly how to apply

regularization and exactly
what regularization means.

Then we'll start to figure out
how to make this work with

our learning algorithms to make

linear regression and
logistic regression,

and in the future,
other algorithms

as well avoid overfitting.

Let's take a look at
that in the next video.