All language subtitles for 14. TF Syntax Basics - Part Two - Creating and Training the Model

Welcome everyone to part two of Caris syntax basics.

In part two we're going to focus on creating the model running our model and then actually generating

predictions from the model.

In part three after this we'll focus on how to actually evaluate the model's performance as well as

how to predict on brand new data sets and save and load our models.

Let's head back to the notebook to continue where we left off last time.

All right.

Here I am at the notebook where we left off last time.

So we've read in our data then just a little bit of analysis on it.

Basically visualize that pair plot and we've scaled our featured data.

So now the next step is to actually create the model with the curious syntax.

And for this we need to do two imports we'll say from tensor flow that carries and this is the way the

curve API is packaged inside of tensor flow.

We just say from tensor flow that carries and then we can do any imports we want as if we had already

installed the caris library separately in the first one we're going to import is the sequential model

and then the next thing we're going to do is say from tensor flow that carries layers will import.

That's and says all we need to build a very simple model with Caris essentially what we do is we set

up a base sequential model and then keep adding layers to it.

In this case we'll just add a simple dense layer.

I would also highly recommend that if you call help on either of these such as help sequential there's

really nice documentation and various examples inside the documentation.

So it actually shows you a lot of what we're going to do which is how to construct the model and then

how to add layers to it et cetera.

So there's lots of really cool examples here not just for sequential but if you also check out dance

it'll give you a lot of information on the various parameters it takes then if you scroll down you'll

eventually also see some examples.

But let's go into work through how we can actually do this.

Now there's two ways of creating a character based model.

One way is to call sequential and then passing a list of the actual layers you want.

So I'm going to pass on a dense layer and if we take a look at dense all this means if we expand on

this is that it's a regular densely connected neural network layer and all that means is if something

is densely connected it's going to be a normal feed forward network where every neuron is connected

to every other neuron in the next layer.

Later on we'll learn about more sophisticated layers but the dense layer is what we're working with

right now for our basic artificial neural networks.

You'll also notice that it has quite a few parameter calls inside of it.

The two parameter calls that we need to be aware of right now is units and activation units is just

another word for neurons.

Essentially how many neurons are actually going to be in this layer and then activation takes in a string

call for what activation function these neurons should be using should they be using a sigmoid activation

they rectified linear unit et cetera.

So for right now what we're gonna do is show you how we could build out a network.

Let's imagine I wanted my first layer to be four neurons densely connected meaning every neurons connected

to every other neuron and then we can say my activation and I could say something like R E L U which

stands for rectified linear unit.

And if you're confused on these activation functions make sure you go back and check out the theory

portions of the section of the course.

Hopefully can see the connection here between what we discussed in theory versus what we're actually

implementing here if code.

And if I wanted to add in another layer I would simply keep passing these into my list so maybe I want

the next layer to have two neurons and another activation functional rectified linear unit.

I could also pass in strings like sigmoid etc. and you can check out the online documentation for the

various string calls for different activation functions.

Let's imagine I wanted one final output layer with one unit.

I would just say that's one again I could play around the activation function there.

But this is just one way we could build out the model.

So that is upon your call for sequential.

You actually pass in a list of those layers the other where we can do this.

And this is going to be our preferred method throughout the course.

You'll see why in a second is to create an empty sequential model and then off that model variable you

add the layers in separately one at a time so you would say activation rectified linear unit and then

I can just copy and paste this command as such and then kind of play around the values here.

Maybe I want this and since this my last layer I don't actually want an activation etc. So this cell

and this cell are actually going to produce the exact same model.

So what's the difference between them.

As far as convenience.

Well what's really convenient about doing this in separate lines like this instead of one giant list

call is if I ever want to quickly edit or turn off a layer I can simply just commented out as so and

then rerun the cell and now I'll go straight from my four neurons to this last one you're on output

layer that's a little harder to do here when we're dealing with a list you would have to kind of delete

this already have it on separate lines and be careful of the way we comment things.

So that's why we're gonna focus on using this methodology for building out our models we'll create an

empty sequential model and then we'll add in the layers one by one.

So let's go ahead and do that for our particular dataset.

In this case what we're gonna do is we'll have three layers with four neurons each using a rectified

linear unit and then our very last layer will be just one final output node.

So the final output layer is actually pretty important and that's gonna be determined by your actual

data and your actual situation of what you're trying to predict.

Recall that with this particular dataset we're predicting a single numerical price value.

So what I want is my very last layer to be a single neuron that produces some sort of price.

So it's going to predict maybe four hundred and fifty dollars or six hundred dollars et cetera.

That's why I'm choosing that final layer to just have dense one where it's just going to try to predict

the price.

So that final output is then going to be measured against the true price.

And we'll do that with some sort of lost function and that's where this final line comes into play which

is compiling your model and compiling your model.

We take a look at shift tab here.

It again has lots of different parameter calls and we're going to be exploring these later on in the

course.

But the main ones we want to look at right now are the optimizer and the loss function.

The optimizer is essentially just kind of asking you how do you actually want to perform this gradient

descent.

Do you want to use as prop or as we discussed there's other methods of optimization such as the atom

optimizer.

So I could also say optimizer is equal to and then in the string past an atom which is the string code

for the atom optimizer and you can reference the documentation to see what optimizes are available for

you and then what's also really important here is this loss parameter and the last parameter that string

code is going to change dependent on what you're actually trying to accomplish here.

And if you take a look at our curves syntax basics notebook if you scroll down we actually have a little

section here on choosing an optimizer and loss.

So if you're performing a multi class classification problem you can really choose a variety of optimizer

is but the loss string parameter you should be calling is categorical cross entropy.

If you're performing a binary classification problem you can again choose various optimizer is but the

loss here will be binary cross entropy and in our case we are performing a regression problem because

our label is a continuous value.

So in our case we will use means squared error as our lost functionality so means square makes sense

because we'll essentially be taking the mean squared error of our predictions against true values and

be trying to minimize that through our optimizer call.

So let's come back to our notebook here and then follow along with that we'll say our optimizer is equal

to our mess prop and more importantly our loss since we're performing a regression based task is MSE

which is our means squared error and once we run that we have a full model ready to go and I'll go ahead

and delete this cell so that we just see that model being created.

So have our models sequential we add in whatever layers we want with as many neurons and activation

functions that we want.

We take care to make sure our last output layer matches the actual task we're trying to solve as well

as when we're compiling it making sure our lost function or loss parameter call matches will we're actually

trying to solve once that is done.

We're ready to train the model or fit the model to the training data and we can do this by saying models

that fit.

And then again if you do shift tab here you'll notice a wide variety of parameters some of which we're

gonna go over in a later lecture.

But the main ones I want to concern you with right now are X Y and then epochs.

So X is simply what are the features that we're training on in this case x train and then why are what

are the actual training labels that correspond to those training feature points which in our case is

white train and then epochs.

What's that stands for.

Is one epoch means you've gone through the entire dataset one time and again as a quick note.

If you take a look at our provided notebook we actually provide essentially a rundown of what this batch

and what epochs mean.

So this epoch is essentially an arbitrary cut off that's defined as one pass over the entire dataset.

So if I've gone through the entirety of X train one time that is one epoch.

So I'll go ahead and say that my model is going to go through the training set.

Two hundred and fifty times.

So two hundred fifty epochs.

Later on we'll discuss had actually choose that number correctly and how we can actually use callbacks

of carers to add an early stopping so that we can just choose some arbitrarily large epochs number and

then our model itself will be smart enough to stop at a certain particular time that is optimized based

off some validation loss.

So keep in mind this is kind of the simplest way we could fit but later all will become more sophisticated

for capabilities and actually fitting our model.

So we'll be dealing with things like batch sizes callbacks validation data etc..

Right now we'll keep it very simple and we'll simply fit on extreme corresponding to why train for two

hundred fifty epochs.

Last thing I want to note here is that there is a verbose call so verbose equals one that essentially

indicates the printed output during training.

So if I run this cell you'll notice that as a continuous training over these epochs it prints out these

little reports for me the higher the number on verbose parameter call.

That means the more information is displayed.

And if you set verbose equal to zero it means it won't actually output anything.

I would recommend however that you set for both equal to some number that is not zero.

So you can get an idea of where you are in your actual training stage.

Otherwise you just see a cell running and you won't know if it's on the tenth epoch or the two hundred

three POC.

Now this is just a very simple and small dataset.

You'll notice that we essentially finished our training and you should also notice that our mean square

error is very large in the beginning because it essentially starts off with just the random weights

and biases.

But as it begins to adjust these weights and biases.

So we'll scroll down to later epochs you'll notice the loss is slowly decreasing and it should decrease

very quickly at first and then kind of slowly as it goes further and further until towards the end it

should begin adjusting to some sort of means squared value so we can go ahead and see this by plotting

it out.

So I'm going to come down here to this new cell and show you how we can actually take a look at this

training history by saying model that history thought history and that returns back.

This dictionary of the corresponding historical losses which means I can pass this to a data frame and

say model that history that history run that I and get this nice data frame so I'll set that as my lost

data frame and then I can actually plot this out by simply saying loss underscored the F plot run that

I see something that looks like this.

So this is very typical of neural network training.

You start with a very high loss during your first couple of epoch runs and then as the weights and biases

start to get adjusted you hopefully see kind of a steady but steep decline in your loss or your error

and eventually it will level off where you're not really doing any sort of improvement in your performance

as you train more and more and later on we'll be able to compare this to our validation loss to actually

check for things like overfishing.

OK so we just went over how to create a model as well as how to fit a model.

Coming up next we're actually going to dive deeper into the evaluation of this model against the test

set as well as how to evaluate against a brand new data point.

So thanks and I'll see you at part 3.