All language subtitles for 004 Preprocessing Categorical Data_en

Speaker: So far, most of what we've seen

were examples of numerical variables,

exchange rates, trading volume, security prices, and so on.

Often though, we must deal with categorical data.

In short, categorical data refers to groups or categories,

such as our cat dog examples,

but the machine learning algorithm

takes only numbers as values, doesn't it?

Therefore, the question when working with categorical data

is how to convert a CAT category into a number

so we can input it into a model or output it in the end.

Obviously, a different number should be associated

with each category, right?

Or better, a tensor, we are getting closer.

Imagine our shop has three products,

bread, yogurt, and muffins.

Now, how do we convert these categories to numbers?

A possible solution could be to enumerate them like this.

Bread equals one, yogurt equals two, muffins equals three.

Unfortunately, this implies there is some order.

It's like saying that a muffin is more than a yogurt,

which is more than bread.

Think about prices.

If we instead had three prices, $1, $2, and $3,

three times $1 is equal to $3.

Using the same logic, does it make any sense to you

that three times bread equals one muffin?

There is another level of ambiguity.

To get from bread to muffins,

we always go through yogurt.

Ultimately, what we have done is assume the data

has some order while it hasn't.

Typically that's an issue

when our data is divided into categories.

Think about the products in a shop,

about different car brands or about people.

So our question becomes how to encode such categories

in a way which will be useful

for a machine learning algorithm.

Two main ways are adopted.

The first one is called One-hot Encoding

and the other Binary Encoding.

We will see how to perform them in the next lesson.

Thanks for watching.