Original subtitles

What happens inside the pipeline function?

In this video, we will look at what actually happens when we use the pipeline function of

the Transformers library. More specifically, we will look at the sentiment analysis pipeline, and

how it went from the two following sentences to the positive labels with their respective scores.

As we have seen in the pipeline presentation, there are three stages in the pipeline. First,

we convert the raw texts to numbers the model can make sense of, using a tokenizer.

Then, those numbers go through the model, which outputs logits.

Finally, the post-processing steps transforms those logits into labels and scores.

Let's look in detail at those three steps, and how to replicate them using the Transformers library,

beginning with the first stage, tokenization. The tokenization process has several steps. First,

the text is split into small chunks called tokens. They can be words, parts of words or punctuation

symbols. Then the tokenizer will had some special tokens (if the model expect them). Here the model

uses expects a CLS token at the beginning and a SEP token at the end of the sentence to classify.

Lastly, the tokenizer matches each token to its unique ID in the vocabulary of the pretrained

model. To load such a tokenizer, the Transformers library provides the AutoTokenizer API.

The most important method of this class is from_pretrained, which will download and cache

the configuration and the vocabulary associated to a given checkpoint. Here, the checkpoint used

by default for the sentiment analysis pipeline is distilbert base uncased finetuned sst2 english.

We instantiate a tokenizer associated with that checkpoint, then feed it the two sentences.

Since those two sentences are not of the same size, we will need to pad the shortest one to

be able to build an array. This is done by the tokenizer with the option padding=True.

With truncation=True, we ensure that any sentence longer than the maximum the model can handle

is truncated. Lastly, the return_tensors option tells the tokenizer to return a TensorFlow tensor.

Looking at the result, we see we have a dictionary with two keys.

Input IDs contains the IDs of both sentences, with 0s where the padding is applied. The second key,

attention mask, indicates where padding has been applied, so the model does not pay attention to

it. This is all what is inside the tokenization step. Now let's have a look at the second step,

the model. As for the tokenizer, there is an TFAutoModel API, with a from_pretrained method.

It will download and cache the configuration of the model as well as the pretrained

weights. However, the TFAutoModel API will only instantiate the body of the model,

that is, the part of the model that is left once the pretraining head is removed.

It will output a high-dimensional tensor that is a representation of the sentences passed,

but which is not directly useful for our classification problem.

Here the tensor has two sentences, each of sixteen tokens and the last dimension is the hidden size

of our model 768. To get an output linked to our classification problem, we need to

use the TFAutoModelForSequenceClassification class. It works exactly as the AutoModel class,

except that it will build a model with a classification head. There is one auto class for

each common NLP task in the Transformers library. Here, after giving our model the two sentences,

we get a tensor of size two by two: one result for each sentence and for each possible label. Those

outputs are not probabilities yet (we can see they don't sum to 1). This is because each model of the

Transformers library returns logits. To make sense of those logits, we need to dig into the third and

last step of the pipeline: post-processing. To convert logits into probabilities, we need to

apply a SoftMax layer to them. As we can see, this transforms them into positive numbers that

sum up to 1. The last step is to know which of those corresponds to the positive or the negative

label. This is given by the id2label field of the model config. The first probabilities

(index 0) correspond to the negative label, and the seconds (index 1) correspond to the positive

label. This is how our classifier built with the pipeline function picked those labels and computed

those scores. Now that you know how each steps works, you can easily tweak them to your needs.