Original subtitles

What is transfer learning?

The idea of Transfer Learning is to leverage the knowledge acquired by a model trained with lots of

data on another task. The model A will be trained specifically for task A. Now, let's say you want

to train a model B for a different task. One option would be to train the model from scratch.

This could take lots of computation, time and data. Instead, we could initialize model B with

the same weights as model A, transferring the knowledge of model A on task B.

When training from scratch, all the model’s weight are initialized randomly. In this example,

we are training a BERT model on the task of recognizing if two sentences are similar or not.

On the left, it’s trained from scratch, and on the right, it’s fine-tuning a pretrained

model. As we can see, using transfer learning and the pretrained model yields better results.

And it doesn’t matter if we train longer, the training from scratch is capped around 70%

accuracy while the pretrained model beats the 86% easily.

This is because pretrained models are usually trained on large amounts of data that provide

the model with a statistical understanding of the language used during pretraining.

In computer vision, transfer learning has been applied successfully for almost ten years.

Models are frequently pretrained on ImageNet, a dataset containing 1.2 millions of photo images.

Each image is classified by one of 1000 labels. Training like this,

on labeled data is called supervised learning. In Natural Language Processing,

transfer learning is a bit more recent. A key difference with ImageNet is that the pretraining

is usually self-supervised, which means it doesn’t require humans annotations for the labels.

A very common pretraining objective is to guess the next word in a sentence,

which only requires lots and lots of text. GPT-2 for instance,

was pretrained this way using the content of 45 millions links posted by users on Reddit. Another

example of self-supervised pretraining objective is to predict the value of randomly masked words,

which is similar to fill-in-the-blank tests you may have done in school.

BERT was pretrained this way using the English Wikipedia and 11,000 unpublished books.

In practice, transfer learning is applied on a given model by throwing away its head, that is,

its last layers focused on the pretraining objective, and replacing it with a new,

randomly initialized, head suitable for the task at hand. For instance, when we fine-tuned a BERT

model earlier, we removed the head that classified mask words and replaced it with a classifier with

2 outputs, since our task had two labels. To be as efficient as possible, the pretrained

model used should be as similar as possible to the task it’s fine-tuned on. For instance,

if the problem it’s to classify German sentences, it’s best to use a German pretrained model.

But with the good comes the bad. The pretrained model does not only transfer its knowledge,

but also any bias it may contain. ImageNet mostly contains images coming from the United States and

Western Europe, so models fine-tuned with it usually will perform better on images from

these countries. OpenAI also studied the bias in the predictions of its GPT-3 model

(which was pretrained using the guess the next work objective). Changing the gender of the prompt

from "He was very" to "She was very" changed the predictions from mostly neutral adjectives

to almost only physical ones. In their model card of the GPT-2 model,

OpenAI also acknowledges its bias and discourages its use in systems that interact with humans.