All language subtitles for 002pipline函数

The pipeline function.

The pipeline function is the most
high-level API of the Transformers library.

It regroups together all the steps to go from raw
texts to usable predictions. The model used is at

the core of a pipeline, but the pipeline also
include all the necessary pre-processing (since

the model does not expect texts, but numbers) as
well as some post-processing to make the output of

the model human-readable. Let's look at a first
example with the sentiment analysis pipeline.

This pipeline performs text classification on a
given input, and determines if it's positive or

negative. Here, it attributed the positive label
on the given text, with a confidence of 95%.

You can pass multiple texts to the
same pipeline, which will be processed

and passed through the model together, as a
batch. The output is a list of individual results,

in the same order as the input texts. Here we
find the same label and score for the first text,

and the second text is judged
positive with a confidence of 99.99%.

The zero-shot classification pipeline is a
more general text-classification pipeline:

it allows you to provide the labels you
want. Here we want to classify our input

text along the labels "education", "politics" and
"business". The pipeline successfully recognizes

it's more about education than the
other labels, with a confidence of 84%.

Moving on to other tasks, the text generation
pipeline will auto-complete a given prompt. The

output is generated with a bit of randomness, so
it changes each time you call the generator object

on a given prompt. Up until now, we have used the
pipeline API with the default model associated to

each task, but you can use it with any model that
has been pretrained or fine-tuned on this task.

Going on the model hub (huggingface.co/models),
you can filter the available models by task.

The default model used in our
previous example was gpt2,

but there are many more models
available, and not just in English!

Let's go back to the text generation pipeline and
load it with another model, distilgpt2. This is

a lighter version of gpt2 created by the Hugging
Face team. When applying the pipeline to a given

prompt, we can specify several arguments, such as
the maximum length of the generated texts, or the

number of sentences we want to return (since
there is some randomness in the generation).

Generating text by guessing the next word in a
sentence was the pretraining objective of GPT-2,

the fill mask pipeline is the pretraining
objective of BERT, which is to guess the value

of masked word. In this case, we ask the two most
likely values for the missing words (according to

the model) and get mathematical or computational
as possible answers. Another task Transformers

model can perform is to classify each word in
the sentence instead of the sentence as a whole.

One example of this is Named Entity Recognition,
which is the task of identifying entities, such as

persons, organizations or locations in a sentence.
Here, the model correctly finds the person

(Sylvain), the organization (Hugging Face) as well
as the location (Brooklyn) inside the input text.

The grouped_entities=True argument used
is to make the pipeline group together

the different words linked to the same
entity (such as Hugging and Face here).

Another task available with the pipeline
API is extractive question answering.

Providing a context and a question, the model
will identify the span of text in the context

containing the answer to the question. Getting
short summaries of very long articles is

also something the Transformers library can
help with, with the summarization pipeline.

Finally, the last task supported by the
pipeline API is translation. Here we use

a French/English model found on the model hub
to get the English version of our input text.

Here is a brief summary of all the
tasks we looked into in this video.

Try then out through the inference
widgets in the model hub!