Original subtitles

1

Hello welcome back to the course on computer vision.

2

Today we're talking about the scale problem the scale problem of the that we have that the is the algorithm

3

actually solves and how it solves.

4

OK.

5

So what is a scale problem.

6

Well let's have a look at this image.

7

This is an image we took in what was in your brain or in the Czech Republic.

8

Yes just some horses at a field.

9

And what can we see here.

10

Well let's just do what we have learned so far.

11

And let's see how the algorithm will go if we just apply the constitution so that all the grid would

12

apply some boxes and right away for them we can see the boxes that have identified have fallen features

13

of horses and boxes which have been found.

14

And so if I remove the green boxes which haven't fallen future forces then you will see that somehow

15

this horse right in front the main horse the one that's like the most obvious to us is humans.

16

It has been missed.

17

The algorithm did not pick up the horse it picked up that horse over there it's got features obviously

18

seen features of that horse's features of that horse and seen pictures of this horse that you can barely

19

see but it does not see this horse.

20

Point blank and once the question is why.

21

Why is that happening here.

22

What's what's going on.

23

Well the reason for that is that this horse is too close.

24

It's too big for the algorithm to pick up features and I'll show you what what I mean by that so let's

25

look at just a pair of rectangles or a couple of rectangles.

26

Look at these ones the largest single out one of them so that rectangle.

27

And while in this grand scheme of things it might seem that how can you not pick up a horse it's pretty

28

obvious that there is a horse.

29

Let's remember what we talked about at the start we said that every single one of these rectangles works

30

as a separate image.

31

It is dealing in its own right with what it has inside and is trying to understand do I see features

32

of a horse or do I not see you if you show a horse and to make it even easier to visualize let's crop

33

the rest of the image and let's look at what this rectangles were getting at.

34

Let's imagine that we are this rectangle.

35

And what this rectangle is seeing is that now by looking at that image can you identify that there's

36

a horse in there.

37

Probably not.

38

What would that look like if anything that looks like.

39

I don't know some like a puddle of mud or maybe I don't know it like some fog very like a fire in the

40

distance.

41

I don't know.

42

Maybe water falling or something like God could be anything right.

43

If you just forget about what we saw previously and you let your eyes adjust.

44

You'll notice that it doesn't resemble a horse in any sort of way.

45

And you'll probably say OK that's fair enough.

46

But there's other features on the horse that you could identify from this image.

47

You know you could maybe identify the who knows of the tail of the eyes.

48

Well let's have a look at that as well.

49

Let's have a look at another rectangle.

50

Look at this one.

51

So here you can see that yes we can kind of identify that there is an eye.

52

And because of the the what what is it called The part that goes in the mouth with the horse.

53

And I should I really should know this but because of that pink strap we can see that it's a horse and

54

and it's fair that the algorithm should be also leveraging that picture.

55

So yes it does.

56

It does resemble a horse better.

57

But remember that the S is not only an to identify that indeed there's a horse in there but it also

58

means to identify it it will need to make a suggestion for the actual box that completes this horse

59

that takes lead it contains a horse.

60

And so from here can you tell just forgetting about that image can you tell where the horse is actually

61

located.

62

Is it is it located like this or is it located like this or maybe it's like the guy is turned his head

63

like this but it's actually standing on this side.

64

Maybe that's one of its hooves Maybe that is the horse's Hooven maybe that the horses standing here

65

so it's very hard for the algorithm to predict what the horse looks like what the full image of the

66

horses are like.

67

Maybe maybe it's maybe there's something else blocking the horse in the way so the horse is so big that

68

in most cases for most rectangles we cannot even see that horse.

69

But even then all those cases where we can Judus certain features identify that it's a horse it's very

70

hard to build the rectangle.

71

And so this is where the S is the the next component that comes to the rescue.

72

And so let's have a look at the scale problem.

73

That is how he deals with the scale problem.

74

So let's move back to the architecture and here we can see how there's many layers.

75

So far we've been talking mostly about looking at examples or one of the original image but in reality

76

what happens is the image region is constantly reduced in size.

77

There are many layers in this network that's working behind the algorithm and or in the background.

78

And basically what it does is it applies a conclusion or operation to reduce the image size so here

79

you can see how it was 300 them it's thirty eight nineteen ten pixels five pixel three pixels one pixel

80

so it's constantly being reduced in size.

81

The distance sensor and and then all of everything we talked about is applied on every single step on

82

all of those images what we were talking about is applied to all those rectangles identified the training

83

of where they should be and what they contain where they should be happens.

84

And as you can see here there's a huge number of detection is eight thousand two hundred and three directions

85

in this set up for 300 300 and put image at the start.

86

Eight hundred eight thousand seven hundred thirty two detections per class.

87

And as you can imagine there's many different classes as well.

88

And that's because there's many iterations and basically what happens is once every time we can involve

89

the image further we are again we are applying these classifiers.

90

Here you can see classifiers are being applied to detect for instance in our case horses again classified

91

as a BNN to apply to detect horses classify as had been applied classify So basically that just shows

92

that everything is being repeated again and again and again.

93

And so let's have a look at that how that happens and then I'll provide some additional comments on

94

that.

95

So there is our image what happens to the image is it is it actually goes through a convolution and

96

we won't you just resize it but that's just for visual purposes just to help us understand better what's

97

going on.

98

In reality it's not just resized.

99

It actually goes through the confusion so the image won't look anything like itself any more it will

100

be completely random looking thing for us but it will preserve the features.

101

So once again we discuss this more in the illusional neural networks tutorials which are in an X number

102

two.

103

But for our purposes just for us to better understand intuition behind things we're going to keep it

104

as an image of horses is the exact image and just should resize that will still help us understand what's

105

going on.

106

And so here what happens is the image is resized but the rectangles are stay the same size and now they

107

are applied to this new image which is resized.

108

And because the image is smaller that means that horse is smaller.

109

And now if we try to pick up features of horses you'll see that these rectangles do pick up a horse

110

whereas those rectangles that used speak of a horse so this horse is not picked up in animal that one

111

isn't that one isn't because they're smaller or not just similar to what we had in the first example

112

with Adlon at the lake and there was people there were people in the background that they were too small

113

to be identified.

114

Same thing here.

115

So just for argument's sake we're going to say that these horses are not picked up and that's totally

116

fine because we're very pick them up before whereas now we've picked up this big horse which in this

117

new version of the image is smaller.

118

And so now it's it will be identified.

119

The question though that still remains is how will we draw the box for this horse in the original image

120

we know from what we discussed in the previous tutorial we know how the algorithm goes about drawing

121

this box.

122

In this version of image because through training it will understand you know it has it has lots of

123

examples of of horses or other objects with the ground truth and will be able to understand how to draw

124

these boxes and so in this case we will also learn how to draw the boxes in this image.

125

But remember that we got this image not just through resizing but through a convolution operation.

126

And that means how do that means we still have a question how do we then take this box and move it back.

127

Well it's pretty straightforward.

128

We're just going to look at our architecture again here and so wherever we are in the process the is

129

the algorithm it's constructed in such a way that it preserves information about how to return how to

130

scale back to the original image so hard to go from let's say this layer to here are hard to go from

131

this layer to here so it all remember that it found the horse for instance over here in this part of

132

the image of this layer and it will also know how it got to this layer so it will know how to get back.

133

So how to decontrol mean in simplistic terms how to get back to this original image and where it should

134

place the horse there so that's an important part that it needs to place the horse in the right place

135

not only in this image but in the original image.

136

And this is the algorithm is equipped to do that.

137

And so basically through this way we are now able to handle objects of different sizes.

138

And the beauty of this is the algorithm is that it doesn't just resize the image and then proceeds do

139

the conversion operation again.

140

The idea here is that all this happens at the same time all of these boxes that we're detecting all

141

of the positions of the boxes the classes inside the boxes and even the resizing or the different scales

142

of the images.

143

It all happens in one neural network over here like this so instead of just having this image and then

144

having a copy of the image where just resize and another copy of the image were smaller and was more

145

up and running neural networks for each one were running all of this in one your own network.

146

And why is that good.

147

Why is that important is because we can utilize the features we can share the features across the different

148

layers so that you know we've when we're when we're training the network it learns how to detect horses

149

when they're in the full size or less so in this in this lair in this size than when they're smaller

150

when they're made smaller and so and so that way all of the layers are learning together.

151

And that gives it that additional power or that you know there's power in numbers that they're they're

152

seeing more different horses at the same time so some horses are seen here some horses are seen here

153

and therefore it is more powerful in the sense that it has more examples to work off and the network

154

can better adjusted its weights to properly detect those horses.

155

So this really helps or facilitates the training.

156

Yeah so that's how the essays the algorithm deals with the scale problem.

157

It's quite a smart solution.

158

I'm sure you'd agree that you know putting all of this into one emotional neural network is is pretty

159

exciting and definitely gives us some amazing results.

160

And then and on that note I hope you enjoyed this tutorial and I look forward to see you next time.

161

And until then enjoy computer vision.