Original subtitles

1

Hello and welcome back to the course and deep learning.

2

Today we're talking about the neuron which is the basic building block of artificial neural networks.

3

So let's get started.

4

Previously we saw an image which looked like this.

5

And these are actual real life neurons which have been smeared onto a gloss and color a little bit and

6

they are observed through a microscope.

7

So this is what they look like as you can see.

8

Quite an interesting structure a body and then lots of different tails kind of branches coming out of

9

them.

10

And this is very interesting.

11

But the question is how can we recreate that in a machine because we really need to recreate it and

12

machine since the whole purpose of deep learning is to mimic how the human brain works in the hope that

13

by doing so we're going to create something amazing.

14

We're going to create an amazing infrastructure for machines to be able to learn.

15

And why do we hope for that.

16

Well because the human brain is well just happens to be one of the most powerful learning learning tools

17

on the planet or like learning mechanisms on the planet.

18

And we just hope that if we recreate that we'll have something as awesome as that.

19

So our challenge right now our very first step to creating artificial neural networks is to recreate

20

a neuron.

21

So how do we do it.

22

Well first of all let's have a closer look at what it actually is.

23

This image was first created by a Spanish neuroscientist and Chagga Ramon Yi Kajal in 1899.

24

And what he did was he died in neurons in actual brain tissue.

25

And look at them under a microscope.

26

And while he was looking at them he actually drew what he saw and this is what he saw.

27

He saw it to your hands or two large neurons over there at the top which had all these branches coming

28

off of them towards their top parts and then each one had this Araud or like thread coming out towards

29

the bottom very long one.

30

And so that's what he saw.

31

And now you know technology has advanced quite a lot and we have seen neurons much closer in more detail

32

and now we can actually draw what it looks like diagrammatic.

33

So let's have a look at that.

34

Here's a neuron.

35

This is what it looks like very similar to what Santiago around one drew over here and here and this

36

year and what we can see is that it's got a body.

37

That's the main part of the neuron and then it's got some branches at the top which is called dendrites

38

and it's also got an X on which is that long tail of the euro.

39

And so what are these dendrites and when foreign was the axen for well.

40

The key point to understand here is that neurons by themselves are pretty much useless.

41

It's like it's like an ant.

42

Right.

43

And on its own can do my psych five ads together.

44

Maybe they can pick something up.

45

But again they they don't they can build an anthill or they call them establish a colony that can't

46

work together as a as a huge organism.

47

But at the same time when you have lots and lots of ads like you have in a million and they can build

48

a whole colony they can build an anthill.

49

Same thing with neurons.

50

By itself it's not that strong but when you have lots of neurons together they work together to do magic.

51

And how do they work together.

52

That's the question.

53

Well that's what the dendrites and Aksenov for so the dendrites are kind of like the receivers of the

54

signal for the neuron and the axon is the transmitter of the signal for the neuron.

55

And here's an image of how it all works conceptually.

56

So at the top you've got on your own and you can see that it's dendrites are connected to axons of other

57

neurons that are like even further away above it.

58

And then the signal from your own travels down its axon and connects or passes on to the dendrites of

59

the next neuron and that's how they're connected.

60

And in that small image over there you can see that the axon doesn't actually touch the dendrite lot.

61

A lot of machine learning or like a few machine learning scientists are very adamant about the fact

62

that it doesn't touch it like the room it doesn't touch.

63

It has been proven that there is no physical connection there.

64

But the point that we're interested in is that that connection between them that the whole concept of

65

the signal being passed that recall the sign ups you can see over there in that little image that's

66

figure bracket is a sign up.

67

And that's the trend we're going to be doing.

68

So instead of calling our artificial neurons the lines that we're going to have or the connectors for

69

artificial neurons we're now going to be calling them axons or dendrites because then the question is

70

whose connection is this is it that neurons are neurons.

71

We just call that good which is good to call them sign of cells.

72

And that's kind of just answers all questions in a.

73

That's just basically where the signal is passed doesn't matter who that element belongs to.

74

They're just a representation of the signal pass and we'll see that just now.

75

So basically that's how a neuron works.

76

And yeah so let's move on to how are we going to represent neuron create neurons in machines are we

77

moving away now we're moving away from neuroscience and moving into technology.

78

And here we go.

79

So here's our neuron also sometimes called the node than your own gets some input signals and it has

80

an output signal.

81

So dendrites and axons remember.

82

But again we're going to call these sign ups and then these input signals we're going to present them

83

of other neurons as well.

84

So in this specific case you can see that this neuron is green you're on is getting signals from yellow

85

neurons.

86

And in this course we're going to try and stick to a certain color coding regime where yellow means

87

an input layer.

88

So basically all of the neurons that are on the outer layer on the first front of where are the signals

89

coming in and by signal.

90

It might be like a bit of an over overkill to call this a signal it's just basically input values so.

91

So you know how even like in a simple linear regression you have input values and then you have a predicted

92

value Same thing here.

93

So you have input values and there they are the yellow ones and then on the right you'll see just now

94

it will be red.

95

It'll be the output value.

96

The other thing that I wanted to point out here is that in this specific example we're looking at a

97

neuron which is getting its signals from the input layer neurons.

98

There are also neurons but their input their neurons.

99

Sometimes you'll have neurons which get their signal from other hidden layer neurons so from other green

100

neurons and the concept is going to be exactly the same I just in this case we use for simplicity's

101

sake we're portraying this example and in terms of the input layer the way to think about it is in the

102

in the analogy of the human brain the input layer is your senses right so whatever you can see hear

103

feel touch or smell.

104

And of course it's like there's there's a lot of things you can see there's a lot of information coming

105

in.

106

But those are your That's what your brain is limited to is pretty much a life.

107

Israel lives in a box made out of bones and it's only just it's it's a mind blowing concept to think

108

about that your brain is just locked in a black box and the only thing it can see can hear.

109

The only thing it's getting is electrical impulses coming from these organs that you have we should

110

call your ears nose eyes you know your sense of touch and we're whatever and you and your and your taste.

111

Right so you're just getting signals but it basically lives in this dark black box and it's making making

112

sense of the world through your senses it's phenomenal.

113

And so yeah.

114

So you have these inputs that are coming in in terms of human brain.

115

Those are your five senses and in terms of machine learning or deep learning that is basically your

116

input values are your independent variables and we'll get that in a second so your input values they

117

the signal is passed through sinuses to the neuron and then your own has an output value that it passes

118

further on down the chain.

119

In this specific case in terms of color coding again yellow means input layer so we kind of simplifying

120

everything here we're saying we're only going to have like the input layer and then we're going to have

121

one hidden layer with the green which is the hinterland then we're going to have the output there right

122

away.

123

So just so that we can get used to these calls for now.

124

So there we go that's the basic structure.

125

So now let's look in a bit more detail at these different elements that we have.

126

So we've got the input layer and what do we have here.

127

Well we have these inputs which are in fact independent variable so depend variable one and a bit variable

128

to independent variable.

129

And the important thing to remember here is that these independent variables are all for one single

130

observation.

131

So think of it as just one row in your database.

132

One observation you just take all of the independent variables you know maybe it's the age of the person

133

there the amount of money in the bank account and then how how do they drive or walk to work what method

134

of Shampoo's protection do they use.

135

So but that's all descriptors of one specific person that you are either your training your model on

136

or you're performing some prediction on.

137

And the other thing you need to know about these variables is that you need to standardize them so you

138

need to either standardize them which means make sure that they have a mean of zero and a variance one

139

or you can also sometimes and headland will point out these traces in a bit more detail perhaps in practical

140

terms you might come across these sometimes you might want to know standardising might want to normalize

141

them.

142

Meaning that instead of making sure the mean and very Muser and variance is one you just take you know

143

to subtract the minimum value and then you divide by the maximum minus minimums by the range of your

144

values and the four you get values between 0 and 1.

145

And it depends on this scenario you might want to do one or the other.

146

But basically you want all of these variables to be quite similar in about the same a range of values

147

and why.

148

Why is that.

149

Well all of these values are going to go into a neural network where as we'll see just now they'll be

150

added up and multiplied by weights it's added up and so on and just going to be is going to be easier

151

for the neural network to process them if they're all about the same.

152

And and in fact you know that's that's just how it is going to be able to work properly.

153

And if you want to read more about standardization normalization and other things that you can do if

154

you know what variables a good additional reading paper is called efficient back prob by young Licken

155

1998 links over there.

156

So we're actually going to talk more about this phenomenal person in the space of deep learning in the

157

part of the course we're talking about.

158

Convolutional neural networks and you'll you'll see that this is definitely a person who knows what

159

he's talking about.

160

He's a close friend of Jeffrey Hinton who we've already seen who very dim..

161

So in this paper you'll learn more about centers ization of normalization but also you can pick up lots

162

of other different tips and tricks and you'll be a good a good source for additional reading as you

163

go through this course.

164

So yeah check it out if you are interested in some additional reading.

165

There we go.

166

So that's what we do for the verbals.

167

And as here we've got the output value.

168

So what can our output value be.

169

Well we've got a couple of options.

170

Output value can be it can be continuous Like for instance price it can be binary for instance a person

171

will exit or will stay or it can be categorical verbal and physical wriggler categorical verbal.

172

The important thing to remember here is that in that case your output value won't be just one it'll

173

be several output values because these will be a dummy variables which will be representing your categories

174

and that's just this how it works and just important to remember that in that case that's how you're

175

going to be getting your categories out of the artificial neural network.

176

But let's go back to a simple case of one output volume.

177

And now let's one more point or kind of like the point the ready made I just wanted to reiterate this

178

point on the left you've got a single observation.

179

So wonder if you mean from your data set and on the right you have a single direction as well and that

180

is the same observation.

181

So important to remember that like whatever inputs you putting in that's for one row and then the output

182

you get that is for that same exact row.

183

Or if you're training your neural network then you know you're putting the inputs in for that one roll

184

you're putting the output in for that one row.

185

So like if you want to simplify the complexity think of it as a like a simple thing a regression or

186

a multivariate linear regression.

187

So you're putting in your values you have the output.

188

There's there's like there's no question about it.

189

When we're talking about things like regression because we're so used to it.

190

Same thing here it's nothing too complex.

191

We're just putting in values we're getting output.

192

But just remember that every time it's one row you're dealing with so you don't get confused and start

193

putting in like thinking that these are different different rows that you're putting into your artificial

194

neural network or something this is all just values in that one Rowse a different observation different

195

characteristics or attributes relating to that one observation every single time.

196

OK so next thing we want to talk about here is our sinuses is Asylum's here we've got services and they

197

all actually get assigned weights weights.

198

We're going to talk more about weights further down but in short weights are crucial to artificial neural

199

network nerve works functioning because weights are how neural networks learn by adjusting the weights

200

the neural network decides in every single case what single signal is poor and what signal is not important

201

to certain neuron.

202

What single gets passed along and what signal doesn't get passed along or what strength to what extent

203

signals get passed along.

204

So weights are crucial.

205

They are and they are the things that get adjusted through the process of learning.

206

Like when when you're training under artificial neural network you're basically adjusting all of the

207

weights in all of the sinuses across this whole neural network.

208

And that's where gradient descent and back propagation come into play and those are concepts that we'll

209

also discuss.

210

So basically those are the weights.

211

That's what I need to know for now.

212

And we've got the neurons so signals go into the neuron and what happens in Europe.

213

So this is the interesting part.

214

Like we're talking about the neuron today what happens inside the neuron.

215

So a few things happen first thing and the first step is that all of these values that it's getting

216

gets added up so it takes that added.

217

So the weighted sum of all of the input values that is getting very simple it's very very straightforward

218

just add up multiply by the way add them up and then it applies an activation function.

219

Now we're going to talk more about activation functions the down but it's basically a function that

220

is assigned to this neuron or to this whole layer and it is applied to this way to add some.

221

And then from that the eurozone understands if it needs to pass on a signal if you like and that's the

222

signal that it passes on that the function applied to the way that some.

223

But basically depending on the function the neuron will either pass on the signal it or it won't pass

224

the signal on.

225

And that's exactly what happened here in step three.

226

The neuron pasta's on that signal to the next neuron down the line.

227

And that's what we're going to talk about in the next tutorial because it is quite an important topic.

228

We want to delve deeper into the activation function but hopefully for now everything is it should be

229

pretty clear how you know the input values you've got weights you've got design houses you've got something

230

you know happens in the neuron you've got a way Sarmad an activation function applied and then that

231

is passed down the line and that is just repeated throughout the whole neural network on and on and

232

on and on you know thousands hundreds of thousands of times depending on how big how many neurons you

233

have how many sign ups as you have in your neural network.

234

So there we go.

235

Hope you enjoyed today's tutorial code to an extent.

236

Until then enjoy deep learning.