All language subtitles for 09 - Training, Validation, and Test Data.en

[Autogenerated] when you're building

machine learning models, it's common

practice to split up the data that you

have to work with into two, maybe even

three subsets training tests and

validation data. Let's talk about why we

do this and how this fits into the machine

learning workflow. We know the basic ___

and board in getting your machine learning

model to production. Observe the step here

where you train your model. Choose a

validation method. Validate your model,

examine and score your model and it read

this process till you're satisfied with

how we're model. Performs when we talk

off, splitting our data into subsets,

these other steps that will use those

subsets of data to validate and evaluate

our model. Let's say this is all the data

that we have available in the world to

feed into our model. This is our entire

world off data. Now let's say you want to

take the view that you want as much data

as possible to train your model. You'll

take all of the state of that. You have

pre process it in the right way and feed

it into building a machine learning model.

But there's a problem with that data that

you used to train a model cannot be used

to evaluate a model. Once you've trained

your mortal, you want to know where that

it's a good one. You can't use the same

training data for that. That's because of

the model has seen all of the instances in

the training data during the process of

trading. So it's quite possible that if

you evaluate your model on the training

data, the model gives you a great score.

But that may not mean that it's a good

model. The model we have memorized

training instances. And if you want to,

indeed uses model on instances that it

hasn't encountered before, it would

perform poorly. It may have over fit on

the training data, so mortal robustness

cannot be measured on instances the model

has encountered before. So if you have all

of the state and the ball and you use

everything to train your model, well, then

you're left with nothing to evaluate your

model. So what do you do now? So you have

all of this data available to you. You

split it into two subsets training data

and test data. It's quite common to use

80% off the data to train the model. This

is referred to as training data, and this

is the only set of instances that your

model we see in the training process you

set aside a portion of the original data.

Let's it's 20% to sanity. Check or measure

the performance off your Marty during the

training process. Your model will never

encounter the test data, which means if

you want to evaluate whether your model is

a robust one, that it works well on

instances it hasn't seen before. Well, you

lose the test data off of that. You will

run through one training process to

generate one candidate model. You'll set

up a model with a certain design, and

you'll get one candidate model at the end

of the training process. If you want any

different candidate models, you'll need to

run in different training processes. And

for each of these end candidate models,

you'll run and test processes toe.

Evaluate these models and then you'll pick

the best one. So far, this is seeming like

a pretty good idea. You have training data

and you have test data. The test data are

the tests that can be used to choose the

best candidate model, which means you're

evaluating our modern instances the model

hasn't seen before. It hasn't seen during

the training process, but there's a

problem here as well. It's quite possible

that your evaluation itself can become

bias when you use the same set of

instances that is the test set toe pick

the best Candidate model. This can lead to

a kind off over fitting. This is refer to

us over fitting on the test set. When you

have data split into just two sets

training and test your candidate model can

end up or fitting on the test it. And once

again, you haven't really got a robust

model her, which leads us to the next

option. Cross validation. Instead of using

just one test set to evaluate your model,

you'll come out a separate validation set

of data points. Now you'll generate

different candidate models using the

training set. You'll then use the

validation set to pick the best candidate

model, and then the final evaluation off.

That model will be on the test set. Let's

take a look at how this works visually,

you have all of the data that you're

working with in the world, you split it

into three subsets. Most of this will be

training data that you used to train the

model. You will have two other subsets

validation data and test data, so you're

holding out to subset from the original

creaming process. So let's say you want to

build a number of different models and

then choose the best one. You lose the

training data to produce the different

candidate models in the news. The

validation data toe. Evaluate all of these

models once you figured out which model is

the best one. Based on the validation

data, you let finally evaluate that

candidate model on the test data. The test

data is comprised of instances that your

model has never seen before. Either during

training are during validation. Test data

gives you an unbiased evaluation off your

model, and with this set up, you can see

whether your finally model is a robust

one. You can also generate multiple

candidate models and select the best one.

This process is called hyper parameter

tuning, and you can evaluate the final

candidate model on your test data so

you'll run and training process is to

generate end candidate models n validation processes and one test process