All language subtitles for 12. The Naive Forecast and the Importance of Baselines

af Afrikaans
ak Akan
sq Albanian
am Amharic
hy Armenian
az Azerbaijani
eu Basque
be Belarusian
bem Bemba
bn Bengali
bh Bihari
bs Bosnian
br Breton
bg Bulgarian
km Cambodian
ca Catalan
ceb Cebuano
chr Cherokee
ny Chichewa
zh-CN Chinese (Simplified)
zh-TW Chinese (Traditional)
co Corsican
hr Croatian
cs Czech
da Danish
nl Dutch
en English
eo Esperanto
et Estonian
ee Ewe
fo Faroese
tl Filipino
fi Finnish
fr French
fy Frisian
gaa Ga
gl Galician
ka Georgian
de German
el Greek
gn Guarani
gu Gujarati
ht Haitian Creole
ha Hausa
haw Hawaiian
iw Hebrew
hi Hindi
hmn Hmong
hu Hungarian
is Icelandic
ig Igbo
id Indonesian
ia Interlingua
ga Irish
it Italian
ja Japanese
jw Javanese
kn Kannada
kk Kazakh
rw Kinyarwanda
rn Kirundi
kg Kongo
ko Korean
kri Krio (Sierra Leone)
ku Kurdish
ckb Kurdish (Soranรฎ)
ky Kyrgyz
lo Laothian
la Latin
lv Latvian
ln Lingala
lt Lithuanian
loz Lozi
lg Luganda
ach Luo
lb Luxembourgish
mk Macedonian
mg Malagasy
ms Malay
ml Malayalam
mt Maltese
mi Maori
mr Marathi
mfe Mauritian Creole
mo Moldavian
mn Mongolian
my Myanmar (Burmese)
sr-ME Montenegrin
ne Nepali
pcm Nigerian Pidgin
nso Northern Sotho
no Norwegian
nn Norwegian (Nynorsk)
oc Occitan
or Oriya
om Oromo
ps Pashto
fa Persian
pl Polish
pt-BR Portuguese (Brazil)
pt Portuguese (Portugal)
pa Punjabi
qu Quechua
ro Romanian
rm Romansh
nyn Runyakitara
ru Russian
sm Samoan
gd Scots Gaelic
sr Serbian
sh Serbo-Croatian
st Sesotho
tn Setswana
crs Seychellois Creole
sn Shona
sd Sindhi
si Sinhalese
sk Slovak
sl Slovenian
so Somali
es Spanish
es-419 Spanish (Latin American)
su Sundanese
sw Swahili
sv Swedish
tg Tajik
ta Tamil
tt Tatar
te Telugu
th Thai
ti Tigrinya
to Tonga
lua Tshiluba
tum Tumbuka
tr Turkish
tk Turkmen
tw Twi
ug Uighur
uk Ukrainian
ur Urdu
uz Uzbek
vi Vietnamese
cy Welsh
wo Wolof
xh Xhosa
yi Yiddish
yo Yoruba
zu Zulu

Original subtitles

In this lecture, we are going to discuss a method of forecasting called the naive forecast before we

discuss what the new forecast is.

Let's talk about the importance of establishing baselines in machine learning.

The purpose of a baseline is to have a relevant point of comparison.

To give you an example, suppose that you went to your professor tomorrow and say, hey, I just made

a great discovery.

I built a model using deep learning and I got 99 percent accuracy.

Unfortunately, the statement is meaningless because you don't have a context of a baseline.

Your professor might say, but student, we already know that a simple linear model achieves 100 percent

accuracy.

In this case, your model is worse than what currently exists.

And furthermore, it's also slower.

So how can we make sure that we don't make this mistake?

The answer is using a baseline.

You'll notice that when you're reading machine learning papers, they often compare their results against

the existing state of the art.

Now, it's important to realize that you don't have to constantly try to beat the state of the art.

In fact, that's kind of antithetical to science because all you're doing is chasing numbers.

A good a real life example of that is when students study very hard to memorize their notes so that

they can ace their exam without truly understanding the material or how it's applied when you're chasing

only numbers.

Sometimes it's easy to forget why those numbers matter in the first place.

Anyway, to get back to the point.

It's not always true that you want to compare to the state of the art.

Sometimes if you just want to test whether or not something is working, for example, as a proof of

concept, then the simplest model possible should suffice.

In Time series analysis, the simplest model possible happens to be the naive forecast, what is the

new forecast?

Well, it's just another name for something we've already discussed that is to copy the previous known

value forward in time.

One interesting phenomena that happens in a Time series analysis is that a lot of bad models end up

looking like naive forecasts.

When you look at the model predictions from afar, they seem to be pretty close to the true values.

But when you zoom in, you can see that they seem pretty close only because they're just copying the

previous value or close to it.

A really bad situation is this suppose that you fit a model and you say, aha, I beat the naive forecast

because my accuracy is better than the Niyi forecast.

But of course, you have to ask, are you talking about the accuracy on the sample data or the out of

sample data?

Note that in machine learning we often refer to these as the training data and the test data.

So I would use these terms interchangeably.

Just be aware.

Well, one common mistake is that people believe because they got good accuracy on their in sample data,

that the same will be true for their out of sample data.

And also, quite commonly, it turns out that the opposite is true.

You might beat the night forecast on the sample data, but on the out of sample data, the naive forecast

beats you.

Why does this happen?

It happens when your model over fits to the noise in the training data, but doesn't actually generalize

to the true underlying pattern in the Time series if one exists.

So that's why it's really important to compare your model to a baseline such as the naive forecast.

It's not good enough to say I got 80 percent classification rate on my train and 75 percent on my test

set.

If some other method achieves 70 percent on the test, then your model isn't as good.

There's one application I've seen a lot, which I think is a very interesting consequence of the times

that we live in today.

There are marketers everywhere on the Internet.

The name of the game is SEO search engine optimization.

Everyone is trying to get clicks for their blog or to get people to sign up for their course or whatever.

And of course, one of the obvious topics that many beginners care about is stock price prediction.

Now, if you're taking this course, then you know better.

But let's continue the story.

So imagine what happens.

You take one of the most popular topics that would appeal to someone who doesn't know about finance

like stock predictions.

You take one of the most popular machine learning algorithms that would appeal to someone who may not

know a lot about machine learning, but has heard many buzzwords like LSM.

LSM has been a very popular model for a sequence model.

And then what do you do?

Well, you combine these two, you make a blog post on stock predictions with LSD.

In fact, many people have done so.

It's obviously a very appealing idea.

No one would blame you for clicking on an article or buying a course that claims to be able to do this.

I won't name any names, but there are some courses that make the very mistake I'm talking about in

this lecture.

Maybe you're watching this and you know where you've seen something like this yourself.

There is another even worse mistake that these marketers make.

But we'll discuss that later when we talk about forecasting in general for a time series.

Models like Arima, if you want, you're encouraged to skip over to the forecasting lecture so we can

continue this discussion.

To get back to the naive forecast, let's recall what we learned about random walks.

As you recall, a random walk is where on every step of a time series I flip a coin or pick a number

from a distribution.

And that number is added to my current position in order to go to the next position.

If my noise distribution is a zero centered Gaussian with variants Sigma Square, which is not unreasonable,

then the best forecast is the naive forecast.

I can do no better than predicting the last known value.

Another way to think about this is if you build a model that you think is good but it cannot beat the

night forecast, then it might suggest that your model is actually worse than a random walk model.

In other words, a random walk model describes the data better than your model.

Can't find what you're looking for?
Get subtitles in any language from opensubtitles.com, and translate them here.