All language subtitles for 03_vectorization-part-2.en

I remember when I first
learned about vectorization,

I spent many hours
on my computer

taking an unvectorized version

of an algorithm running it,

see how long it run,
and then running

a vectorized version of the code

and seeing how much
faster that run,

and I just spent hours
playing with that.

And it frankly blew my mind that

the same algorithm vectorized
would run so much faster.

It felt almost like
a magic trick to me.

In this video, let's figure

out how this magic
trick really works.

Let's take a deeper look at how

a vectorized implementation may

work on your computer
behind the scenes.

Let's look at this for loop.

The for loop like this runs
without vectorization.

If j ranges from 0 to say 15,

this piece of code performs

operations one after another.

On the first timestamp which
I'm going to write as t0.

It first operates on
the values at index 0.

At the next time-step,

it calculates values
corresponding to index

1 and so on until the 15th step,

where it computes that.

In other words, it calculates

these computations
one step at a time,

one step after another.

In contrast, this
function in NumPy is

implemented in the computer
hardware with vectorization.

The computer can get all
values of the vectors w and x,

and in a single-step,

it multiplies each pair of w and

x with each other all at
the same time in parallel.

Then after that, the computer
takes these 16 numbers and

uses specialized hardware to

add them altogether
very efficiently,

rather than needing to carry
out distinct additions

one after another to add
up these 16 numbers.

This means that codes with
vectorization can perform

calculations in much less time

than codes without
vectorization.

This matters more when
you're running algorithms on

large data sets or trying
to train large models,

which is often the case
with machine learning.

That's why being able to
vectorize implementations

of learning algorithms,

has been a key step to getting

learning algorithms
to run efficiently,

and therefore scale well
to large datasets that

many modern machine
learning algorithms

now have to operate on.

Now, let's take a look at

a concrete example of
how this helps with

implementing multiple
linear regression

and this linear regression
with multiple input features.

Say you have a problem with

16 features and 16 parameters,

w1 through w16,

in addition to the parameter b.

You calculate it 16
derivative terms

for these 16 weights and codes,

maybe you store the values
of w and d in two np.arrays,

with d storing the values
of the derivatives.

For this example, I'm

just going to ignore
the parameter b.

Now, you want to
compute an update

for each of these 16 parameters.

W_j is updated to w_j
minus the learning rate,

say 0.1, times d_j,

for j from 1 through 16.

Encodes without vectorization,

you would be doing
something like this.

Update w1 to be w1 minus

the learning rate
0.1 times d1, next,

update w2 similarly,

and so on through w16,

updated as w16 minus
0.1 times d16.

Encodes without
vectorization, you can use a

for loop like this
for j in range 016,

that again goes from 0-15,

said w_j equals w_j
minus 0.1 times d_j.

In contrast, with factorization,

you can imagine the computer's

parallel processing
hardware like this.

It takes all 16 values in

the vector w and
subtracts in parallel,

0.1 times all 16 values
in the vector d,

and assign all 16 calculations

back to w all at the same
time and all in one step.

In code, you can implement
this as follows,

w is assigned to w minus 0.1
times d. Behind the scenes,

the computer takes these
NumPy arrays, w and d,

and uses parallel
processing hardware to

carry out all 16
computations efficiently.

Using a vectorized
implementation,

you should get a much more
efficient implementation

of linear regression.

Maybe the speed
difference won't be huge

if you have 16 features,

but if you have thousands of

features and perhaps very
large training sets,

this type of vectorized
implementation will make

a huge difference in

the running time of your
learning algorithm.

It could be the
difference between codes

finishing in one or two minutes,

versus taking many hours
to do the same thing.

In the optional lab that
follows this video,

you see an
introduction to one of

the most used Python libraries
and Machine Learning,

which we've already
touched on in

this video called NumPy.

You see how they create
vectors encode and

these vectors or lists of

numbers are called NumPy arrays,

and you also see how to
take the dot product of

two vectors using a NumPy
function called dot.

You also get to see

how vectorized code such
as using the dot function,

can run much faster
than a for-loop.

In fact, you'd get to
time this code yourself,

and hopefully see
it run much faster.

This optional lab introduces

a fair amount of
new NumPy syntax,

so when you read through
the optional lab,

please still feel
like you have to

understand all the
code right away,

but you can save this notebook
and use it as a reference

to look at when
you're working with

data stored in NumPy arrays.

Congrats on finishing this
video on vectorization.

You've learned one of the most

important and useful techniques

in implementing machine
learning algorithms.

In the next video,

we'll put the math of

multiple linear regression
together with vectorization,

so that you will influence
gradient descent for

multiple linear regression
with vectorization.

Let's go on to the next video.