Original subtitles

Welcome back everyone to this lecture on the perception model to begin understanding deep learning.

We're gonna start by actually building up our model abstractions.

We're going to begin by exploring what real single biological neurons look like.

Then we'll model those mathematically as a single perception see how we can group together perceptions

or artificial neurons to build out a multilayer perception model and then see how that expands to a

deep learning neural network framework so as we learn more about complex models we also then introduce

some key mathematical concepts to the actual mathematical functionality of those neurons.

And that is things like activation functions gradient descent and back propagation.

But for now we're gonna focus on just building out that single precept Tron model based off a real biological

neuron so if the whole idea behind deep learning is to have computers artificially mimic biological

and natural intelligence it's probably a good idea that we should build a general understanding of how

biological neurons work so here we can see some real state neurons in a cerebral cortex.

And basically what we're going to attempt to do is check out the way these biological neurons work and

see if we can develop a simplified abstraction of these real neurons.

So here's a illustration of biological neurons.

You can see that there is a lot going on and don't worry.

This isn't a biology class.

And the main things we're going to focus on is just the neuron itself.

So the main things to see here is that there is some sort of nucleus or center of this neuron There's

the cell body and it looks like we have both inputs and outputs or axons spreading out from this nucleus

so let's take this illustration of a biological neuron and we're we're gonna do is we're going to really

simplify this we're gonna simplify it so much that I will actually create my own illustration and I'm

definitely not an artist.

It's an extremely kind of rough illustration but the worry you'll see why draw it like this later on

in this lecture.

But the main thing you want to focus off the biological opponents are the dendrites and we can think

of those as input going into the main nucleus of this neuron and then the axon which is some sort of

output.

Now again this isn't 100 percent biologically correct but this feeds into our perception model.

Basically we're gonna think of neurons biologically as accepting some sort of input signal and the input

signal can come from a variety of sources and then we'll do some sort of calculation the nucleus or

something happens in the nucleus of this neuron and then it outputs as a single output at Axon.

And that can then lead over to another nucleus as that then right to another neuron.

Now how do we take this very simplified biological neuron model and convert it into a mathematical model.

This is where the idea of the perception comes in a perception was a form of neural network introduced

all the way back in 1958 by Frank Rosenblatt.

And amazingly even back then in the 1950s he saw huge potential stating quote a perception may eventually

be able to learn make decisions and translate languages.

And if you have ever used something like Google Translate Google Translate actually uses a neural network

as its main way of converting from one language to another.

And Frank Rosenblatt actually saw the perception as one of the main base units to building out that

sort of technology.

And he thought of this all the way back in the late 1950s however about a decade later in the late 1960s

two researchers Marvin Minsky and Seymour paper they published a book labeled perceptions and it suggested

that there were actually severe limitations to what perceptions would be able to do and a big part of

that limitation was the actual computational power necessary to work with multilayer perception models.

So back in the late 1960s there wasn't really enough computational power to actually make full use of

the idea of the perceptual mathematical model.

And this publication this book actually marks the beginning of what's known as the A.I. winter where

very little funding went into research in artificial intelligence and neural networks in the 1970s.

So that's why even though this idea was spawned in the late 1950s it's taken up to the modern age to

actually fully implement neural networks now fortunately for us we already now live in the present and

we know the amazing power of neural networks which all really stem from this simple perceptual model

that was created all the way back in 1958.

So let's go ahead and head back out and convert that simple biological neuron model into the perception

model and then go through a simple example of how a perception model works so recall we have a very

simplified biological neuron model.

So this is the biological neuron model than a very simple fashion we have incoming then right then some

sort of center nucleus and a single output to the axon.

So what we're going to do is instead we're going to replace these biological units with some mathematical

ones.

So we're going to do is we're going to define some set of inputs going into this single point which

we'll call the neuron and then has that single output so let's go ahead and work through a simple example

of how the simple perception model actually works.

What we're gonna do is we're gonna imagine essentially two data points X1 next to so these two variables

X1 next to are going into as input into this perception and you could imagine that inside this perception

that neuron is going to have some sort of function.

So it's going to have some sort of functionality and perform something on those inputs of X and it's

going to then output some y.

So we have some function inside the neuron takes in these x values perform something on them and then

outputs y so if ever X that function is just the sum then Y is equal to X1 plus X2 and this is a very

very simplified perception model.

Later on we'll expand on this with activation functions.

But right now this is just showing you how we can go from a biological neuron to a perceptual model.

So we have these inputs X1 next to we push them through some sort of function of x and then we output

some y so realistically we would want to be able to adjust some parameter in order for the perception

to learn.

Right now there's no real way for the perception to adjust its ability to learn to kind of correct the

output of Y.

So what we can do is we can add an adjustable weights that will multiply against these inputs of x.

So we're going to say here is that we'll take in each input of X and apply it to its own weight and

then we can adjust that weight as necessary in order to get the correct value of y or the expected value

of y that maybe we're performing some sort of supervised learning on.

So maybe we're trying to solve a regression task where y is the actual continuous label we're trying

to predict or maybe we're doing classification and Y is a category.

But the most important part here is we're gonna need to be able to take this perceptual model and adjust

it somehow in order to get the correct value of y and one way we can do this is by applying a weight

to each input and then we can adjust the weight.

So we'll label these weights corresponding to their inputs as w 1 for x 1 and w 2 4 x 2 which means

if our function still simplified is just an addition we'll say Y is equal to x 1 times w one plus x

2 times l B2 now we could update the weights in order to affect the output y but what happens if X input

is actually zero.

That means no matter what you do to the weight that actually won't change anything.

So to fix that problem we could add in a bias term B to the inputs.

So regardless if x 1 is 0 or x 2 a zero.

Which means that when you multiply it by a weight it always end up being zero regardless of how you

update the weight.

Well we can do is we can add in a bias term B to the inputs so we essentially assign this neuron its

own particular bias so that the inputs end up not just getting multiplied by a weight but having a bias

added to them.

And keep in mind the weights can be positive or negative and the biases can also be positive or negative.

A good way to think about the bias is that the multiplication or the product of X 1 times w 1 or the

input times its weight has to overcome the bias value in order to start having an effect on the output

of Y so here for this very simple example we end up approximating the output of Y to be equal to x 1

times they'll be 1 plus B plus x 2 times they'll be 2 plus B and is just a very simple summation that's

happening inside the neuron later on we'll talk about more realistic activation functions that you'll

have in typical networks.

Keep in mind we can expand this to a generalization for a variety of inputs all the way to some input

and and so now we've been able to see how we can model a biological neuron as a simple perception MF

perfectly.

Our generalization was simply starting at AIS equal to one all the way to any number of inputs we simply

grabbed that particular input value multiply it by some weight that later on we'll be adjusting plus

a bias value later on we're actually going to see how we can expand this model to have X be a tensor

of information right tensor is an end dimensional matrix so let's review what we've learned so far we

understand the very very basics of a biological neuron the fact that there's dendrites providing input

to a nucleus and then some Axon as a single output and what we've been able to do is take the research

done in the late 1950s of the simple perceptual model and create a very simple mathematical model replicating

those core concepts behind a neuron where we have inputs we apply some sort of weight to them as well

as adding a bias term pass them through some sort of function and then we get out that single output

so let's go ahead and then in the next lecture learn how we can expand a single perception to a full

neural network and then we'll see how we can add in more complex ideas such as activation functions

cost functions and then see how that works of a feed for or network as well as back propagation.

OK thanks and I'll see you at the next lecture.