Original subtitles

In this video, you see

a very useful idea called vectorization.

When you're implementing a learning algorithm,

using vectorization will both make your code

shorter and also make it run much more efficiently.

Learning how to write

vectorized code will allow you to also

take advantage of

modern numerical linear algebra libraries,

as well as maybe even GPU

hardware that stands for graphics processing unit.

This is hardware objectively designed

to speed up computer graphics in your computer,

but turns out can be used when you write vectorized code

to also help you execute your code much more quickly.

Let's look at a concrete example

of what vectorization means.

Here's an example with parameters w and b,

where w is a vector with three numbers,

and you also have a vector of features

x with also three numbers.

Here n is equal to 3.

Notice that in linear algebra,

the index or the counting starts from 1 and so

the first value is subscripted w1 and x1.

In Python code,

you can define these variables w,

b, and x using arrays like this.

Here, I'm actually using

a numerical linear algebra library

in Python called NumPy,

which is by far

the most widely used numerical linear algebra library

in Python and in machine learning.

Because in Python,

the indexing of arrays while

counting in arrays starts from 0,

you would access the first value of

w using w square brackets 0.

The second value using w square bracket 1,

and the third and using w square bracket 2.

The indexing here, it goes from 0,1 to

2 rather than 1, 2 to 3.

Similarly, to access individual features of x,

you will use x0,

x1, and x2.

Many programming languages including Python

start counting from 0 rather than 1.

Now, let's look at an implementation without

vectorization for computing the model's prediction.

In codes, it will look like this.

You take each parameter

w and multiply it by his associated feature.

Now, you could write your code like this,

but what if n isn't three but instead n is a 100 or a

100,000 is both inefficient for you

the code and inefficient for your computer to compute.

Here's another way. Without using

vectorization but using a for loop.

In math, you can use

a summation operator to add all the products of w_j

and x_j for j equals 1 through

n. Then I'll cite the summation you add b at the end.

To summation goes from j equals 1 up

to and including n. For n equals 3,

j therefore goes from 1, 2 to 3.

In code, you can initialize after 0.

Then for j in range from 0 to n,

this actually makes j go from 0 to n minus 1.

From 0, 1 to 2,

you can then add to f the product of w_j times x_j.

Finally, outside the for loop, you add b.

Notice that in Python,

the range 0 to n means that j goes from 0

all the way to n minus 1 and does not include n itself.

This is written range n in Python.

But in this video, I added a 0

here just to emphasize that it starts from 0.

While this implementation is

a bit better than the first one,

this still doesn't use factorization,

and isn't that efficient?

Now, let's look at how you can

do this using vectorization.

This is the math expression of the function f,

which is the dot product of w and x plus b,

and now you can implement this

with a single line of code by

computing fp equals np dot dot,

I said dot dot because the first dot is the period and

the second dot is the function or the method called DOT.

But is fp equals np dot dot w comma x and

this implements the mathematical dot products

between the vectors w and x.

Then finally, you can add b to it at the end.

This NumPy dot function is a vectorized implementation of

the dot product operation between

two vectors and especially when n is large,

this will run much faster

than the two previous code examples.

I want to emphasize that vectorization

actually has two distinct benefits.

First, it makes code shorter,

is now just one line of code. Isn't that cool?

Second, it also results in your code running much faster

than either of the two previous implementations

that did not use vectorization.

The reason that the vectorized implementation

is much faster is behind the scenes.

The NumPy dot function is

able to use parallel hardware in

your computer and this is true whether

you're running this on a normal computer,

that is on a normal computer CPU

or if you are using a GPU,

a graphics processor unit,

that's often used to accelerate machine learning jobs.

The ability of the NumPy dot function

to use parallel hardware makes it much more

efficient than the for loop or

the sequential calculation that we saw previously.

Now, this version is much more

practical when n is large because you are

not typing w0 times x0 plus w1 times x1

plus lots of additional terms

like you would have had for the previous version.

But while this saves a lot on the typing,

is still not that computationally

efficient because it still doesn't use vectorization.

To recap, vectorization makes your code shorter,

so hopefully easier to write

and easier for you or others to read,

and it also makes it run much faster.

But honest, this magic behind

vectorization that makes this run so much faster.

Let's take a look at what your computer is actually doing

behind the scenes to make

vectorized code run so much faster.