Original subtitles

So in this lecture, we'll be looking at how to do fine tuning for text classification when we have

more than one input sentence.

We'll begin by installing transformers and data sets, as we normally do.

The next step is to import low data set as well as numpy.

The next step is to load in our dataset, which is the RTC dataset.

This is part of the glue benchmark.

Like the earlier example on sentiment analysis.

Note that I've pasted a description of the dataset here which explains that the data has been processed

to be for binary classification.

Other options are possible, for example, having three classes where the third class is neutral, meaning

that the two sentences neither entail nor contradict.

The next step is to print out the data set object.

We got back to see what it contains.

So as you can see, we have a train of validation and test sets.

Each data set object contains two sentence columns along with the label and index.

The next step is to check out the features attributes.

So as you can see, this tells us that we have two classes where the first classes entail men and the

second is not entitlement.

The next step is to simply print out some sentences just to see what we're dealing with.

So feel free to check these out, if you like.

The next step is to define our checkpoint.

As mentioned, will be using distilled word for this notebook.

But feel free to try it as well as an exercise.

The next step is to import everything we need from the Transformers library, including the tokenized

model trainer and training arguments.

The next step is to load up our tokens there from the checkpoint.

The next step is to test their token either on the first pair of sentences in our dataset.

The next step is to check the keys of the dictionary.

We got back.

So as you can see, the key for token type IDs is not present.

Recall that one exercise for this notebook is to use Bert instead and to check out the format of the

token type.

It is.

The next step is to decode the input IDs.

So as you can see, our input text is effectively the two sentences concatenated into a single string.

We use the set token to separate the two sentences.

The next step is to load up our Pre-Trained model from the checkpoint.

The next step is to create the training arguments object.

Note that there is one new argument here which is logging steps.

The reason we need this is because this data set doesn't have a lot of samples.

However, the default value for logging steps is very large, resulting in this operation not occurring.

The result is that without setting this argument, we would see no log appearing under the train laws.

When we train our model instead of the loss, which is nice to know.

So try commenting out this argument to see that for yourself.

The next step is to import the load metric function.

The next step is to get the metric for our current task.

The next step is to test our metric just to see what we will get back.

So unfortunately we only get accuracy, which is kind of boring.

Instead we're going to include the F1 score, so we're going to import F1 score from psyche to learn.

The next step is to define our compute metrics function.

We begin by splitting up the logic and the labels.

The next step is to take the arguments of the logic s in order to get the predictions.

Once we have the predictions, we can then compute the accuracy and the F1.

Finally, we return a dictionary containing our desired metrics.

The next step is to define our tokenize, our function as usual.

The input to this function is a batch of data from our datasets.

We can pass in both sets one and send it to a separate arguments and also truncation equals true.

The next step is to create our tokenized datasets.

The next step is to create our trainer object.

The next step is to call trainer to train.

Okay.

So notice that five bucks is probably too much since the validation loss seems to be creeping up.

So as a final exercise for this lecture, don't forget to try Bert with this notebook as well.

Compare the performance of the two models as well as the training time.

Consider if this were a real world project, which model you would prefer to use?

As another exercise.

Please also compute the metrics on our test set as well.