Original subtitles

1

What Allison should know what are you saying I don't know what Internet is that massive computer.

2

Was the one that's becoming really big now.

3

What do you mean.

4

That's just what you know what do you write to what like I don't know a lot of people use it and communicate.

5

I guess they can communicate with NBC writers and producers.

6

Allison can you explain what Internet is

7

how amazing is that.

8

Just over 20 years ago people didn't even know what the internet was.

9

And today we can't even imagine our lives are for it.

10

Welcome to the people earning ETAs that course.

11

My name is Cora Menko and along with the contractor had Lunda Pontmercy were super excited to have you

12

on board.

13

And today we're going to give you a quick overview of what deploring it is and why it's picking up right

14

now.

15

So let's get started.

16

Why did we have a look at that clip and what is this photo over here.

17

Well that clip was from 1994.

18

This is a photo of computer from 1980.

19

And the reason why we kind of delving into history a little bit is because neural networks along with

20

deep learning have been around for quite some time and they've only started picking up now and impacting

21

the world right now.

22

But if you look back at the 80s you'll see that even though they were invented in the 60s and 70s they

23

really caught on to a trend or called the cold wind in the 80s so people are talking about them a lot.

24

There was a lot of research in that area and everybody thought that deep learning or neural networks

25

were this new thing that is going to impact the world is going to change everything is going to solve

26

all the world problems and they did kind of slow they died off over the next decade.

27

And so what happened why did why did the neural networks not survive and not change the world with it

28

was it.

29

The reason for that that they were just not good enough that they are not that good at predicting things

30

are not that good at modeling and messy just not a good invention.

31

Or is there another reason.

32

Well actually there is another reason and the reason is in front of us it's the fact that technology

33

back then was not up to the right standard to facilitate neural networks in order for neural networks

34

and deep learning to work properly.

35

You need two things you need data and you need a lot of data and you need processing power you need

36

strong computers to process that data and facilitate and you know that works.

37

So let's have a look at how as data or storage of data has evolved over the years and then we'll look

38

at how technology has evolved.

39

So here we go got three years 1956 1980 2017.

40

How much did storage look back in 1956.

41

Well there's a hard drive and that hard drive is only a five.

42

Wait for a megabyte harddrive That's five megabytes right there on the forklift the size of a small

43

room that's a hard drive being transported to another location on a plane.

44

And that is what storage looked like in the.

45

In 1956 you had to pay a company had to pay two and a half thousand dollars of those days dollars to

46

rent that hard drive to rent it not buy it or rented for one month.

47

In 1980 the situation improved a little bit.

48

So here we got a 10 megabyte hard drive for three and a half thousand dollars is still very expensive

49

and only 10 megabytes So that's like one photo these days.

50

And today in 2017 we've got a 256 gigabyte SD card for $150 which can fit on your finger.

51

And if you're watching this video a year later or like in 2019 or 2025 you probably laughing to us all

52

because by then you have even stronger storage capacity.

53

But nevertheless the point stands if we compare these across the board and we even taking price and

54

size into consideration just the capacity of whatever was trending at the time.

55

So from 1956 to 1980 capacity increased about double.

56

And then it increased about twenty five thousand six hundred times and the know the length of the period

57

is not that different from 1956 to 1980 24 years from 1980 to 2013 thirty seven years so not that much

58

of an increase in time but a huge jump in technological progress.

59

And that stands to show that this is not a linear trend.

60

This is an exponential growth in technology and if we add into it take into account price and size you

61

will be in the millions of increase.

62

And here we actually have a chart on a logarithmic scale.

63

So if we plot the harddrive cost per gigabyte you'll see that looks something like this.

64

We're very quickly approaching zero.

65

Right now you can get storage on Dropbox and Google Drive which doesn't cost you anything.

66

Cloud storage and that's going to continue and in fact over the years this is going to go even further.

67

Right now scientists are looking into using DNA for storage.

68

And right now it's quite expensive it costs $7000 to synthesize two megabytes of data and then another

69

thought 2004's to read it.

70

But that kind of reminds you of this whole situation of the harddrive and the plane you know that this

71

is going to be mitigated very very quickly with this exponential curve 10 to 10 years from now 20 years

72

from now everybody's going to be using DNA storage if we go down this direction.

73

And here are some stats on that so you can explore it further maybe pause the pause the video if you

74

want to read a bit more about this.

75

This is from nature dot com.

76

And basically you can store all of the world's data in just one kilo one kilogram of DNA storage or

77

you can store about 1 billion terabytes of data in one gram of DNA storage.

78

So that's just something to to show how quickly we're progressing and that this is why deep learning

79

is picking up now that we are finally at the stage where we have enough data to train super cool super

80

sophisticated models.

81

Back then in the 80s when I was first initially invented juice just wasn't the case.

82

And the second thing we talked about is processing capacity So here we've got an exponential curve again

83

on a log scale.

84

It's an ideally portrayed here but on the right because it's a log scale and this is how computers have

85

been evolving.

86

So again feel free to post the slide this is called Moore's Law you've probably heard of it how quickly

87

the processing capacity of computers has been evolving.

88

Right now we're somewhere over here where an average computer can buy for a thousand bucks thinks at

89

the speed of the brain of a rat and between two and five will be the speed of a human or 20:23 and then

90

by 2050 or 2045 it will surpass all of the humans combined.

91

So basically we're entering the era of computers that are extremely powerful that can process things

92

WAY faster then then we then we can imagine.

93

And that is what is facilitating the learning.

94

So all of this brings us to the question What is deep learning what what is this whole neural network

95

situation what what is going on what are we even talking about here.

96

And you've probably seen a picture or something like this.

97

So let's dive into it.

98

What is deep learning this gentleman over here Jeffrey Hinton is known as the godfather of deep thing.

99

And he did research on deep learning in the 80s and he's done lots and lots of work lots of research

100

papers he's published in deep learning right now.

101

He works at Google.

102

So a lot of the things that we're going to be talking about actually come from Jeffrey Hinton and you

103

can see a lot.

104

He's got quite a few YouTube videos.

105

He explains things really well so I highly recommend checking them out.

106

And so the idea behind deep learning is to look at the human brain.

107

And this guy is going to be quite a bit of neuroscience coming up.

108

And in these tutorials and what we're trying to do here is to mimic how the human brain operates.

109

And you know we don't know that much you don't know everything about the human brain but that little

110

man that we all know we want to mimic it and recreate it.

111

And why is that.

112

Well because the human brain seems to be one of the most powerful tools on this planet for learning

113

for learning adapting skills and then applying them and if computers could copy that then we could just

114

leverage what natural selection has already decided for us.

115

All of those kind of algorithms that it has decided are the best which are going to leverage that.

116

Why reinvent the bicycle ride.

117

So let's see how this works.

118

Here we've got some neurons so these neurons which have been smeared onto glass and then have been looked

119

at under a microscope with some coloring.

120

And this is you can see what they look like so they have like a body they have these branches and they

121

have like tails and so and so you can see them they have like a nucleus inside in the middle and that's

122

that's basically what a neuron looks like in the human brain.

123

There's approximately 100 billion neurons all together so these are individual neurons these are actually

124

motor neurons because they're bigger they're easier to see but nevertheless there's a hundred billion

125

neurons in the human brain.

126

And it is connected to as many as about a thousand of its neighbors.

127

So to give you a picture this is what it looks like.

128

This is an actual data section of the human brain.

129

And this is the cerebellum which is this part of your brain at the back.

130

It is responsible for like more Torex and for you know keeping a balance and some language capabilities

131

and something like that.

132

So this is just to show how Vorst How many neurons there are like billions and billions and billions

133

of neurons all connecting It's like we're talking about five or five hundred or a thousand or millions

134

billions of neurons in there.

135

And so that's that's what we're going to be trying to recreate.

136

So how do we recreate this in a computer.

137

Well we create an artificial structure called an artificial neural net where we have nodes or neurons

138

and we're going to have some neurons for input value so these are values that you that you know about

139

a certain situation.

140

So for instance you're modeling something you want to predict something you always could have some input

141

something to start.

142

Your prediction is off then that's called the input layer.

143

Then you have the output.

144

So that's of value that you want to predict or it's surprise whether it's is somebody going to leave

145

the bank or stay in the bank.

146

Is this a fraudulent transaction it's a real transaction and so on.

147

So that's going to be output lower.

148

And in between we're going to have a hidden layer.

149

So as you could see in your brain you have so many neurons so some information is coming in through

150

your eyes ears nose so basically your senses.

151

And then it's it's not just going right away to the output where you have the result is going through

152

all of these billions and billions and billions of neurons before guess output and this is the whole

153

concept behind it that we're going to model the brain.

154

So we need these hidden layers that are there before the output so the input Lares neurons connected

155

to a hidden layer neurons that neurons are connect to output.

156

And so this is this is pretty cool.

157

But what is this all about.

158

Where is the deep learning here or why is it called deeper nothing deeper in here.

159

While this is kind of like an option which one might call shallow learning where there isn't much indeed

160

going on.

161

But why is it called deploring.

162

Well because then we take this to the next level we separate it even further and we have not just one

163

hit and there we have lots and lots and lots of hidden layers and then we connect everything just like

164

in the human brain connect everything interconnected everything.

165

And that's how the input values are processed through all these hidden layers just like in the human

166

brain.

167

Then we have an output value and now we're talking deep learning.

168

So that's what learning is all about on a very abstract level.

169

And the further tutorials we're going to dissect and dive deep into deep learning and by the end of

170

it you will know what the planning is all about and you will know how to apply it in your projects.

171

Super excited about this wait to get started and I look forward to seeing the next tutorial.

172

Until then enjoy deep learning.