Original subtitles

1

Hello and welcome to this tutorial.

2

Now we have everything we have our frames that we're going to get from the video.

3

We have our neural network net the SS The neural network and we have our transform transformation.

4

So we are ready to do some object detection on a video.

5

This video is going to be the funny dog that before there is this video of this very cute dog bouncing

6

on the field on the grass there is curial in the video.

7

We're going to try to detect as well another human and some other humans behind.

8

Let's see if it's powerful enough to even detect the humans that are behind the yard.

9

Well we might figure it out in this Statoil.

10

And if not in this detail it's going to be the next one so let's do it.

11

Let's start by opening the video.

12

That's the first thing we need to do.

13

Then we're going to get all the frames of the video one by one.

14

We're going to apply the detect function on these frames with our SSD net and our transform transformation.

15

Then we'll get the processed images with this rectangle and then we'll reassemble the whole thing to

16

have the final video.

17

All right let's do this let's first open the video.

18

So we're going to create a new object that we're going to call reader and this object is going to be

19

created with Image IO image.

20

I always a great library to process videos.

21

There is another great library that could do the job that is Bill P L in capital letters.

22

We actually tried that but it turned out to be much more efficient with Image IO.

23

So we're going to open the video with Image IO and to do this we well first we get our image I O library

24

and then we're going to use to get this core reader function.

25

And inside this function what do we need to input.

26

Well of course it's the video end quote and the video is well the name of the video is funny dog Dutt

27

and for.

28

All right.

29

So that opens the video basically funny dog that and before we're going to watch the video again before

30

we get the final output then the next step is to get the frequence of the frames.

31

That is the FPL frequents.

32

FPL means frames per second and we just need to get this frequence because we're going to need it afterwards.

33

So let's call this frequence fix and introducing a new variable and to get it.

34

Well we can get it from our reader object from which we used to get underscore Meda underscored data

35

Barondess is nothing inside.

36

But then in square brackets here you have to specify in quotes.

37

US and that will just get you the FBI frequence that is the number of frames per second.

38

All right.

39

And now next step and you're going to understand now why we needed that frames per second frequents

40

the next step is to create and now put video that is going to be the final output with that same FBA

41

sequence.

42

And we're going to create that output video with Again image IO.

43

So we have to give it a different name we're going to call it writer and again and we're going to call

44

our image I O library.

45

And this time since we're not opening a video we are creating a new video where we're not going to use

46

a get rid of function.

47

We're going to use to get writer function that basically creates something like an object that will

48

contain a video.

49

And this something will add the sequence of frames you know we will append to process frames that is

50

the frames on which we apply to detect function.

51

So there we go get writer.

52

And then we need to put two arguments the first one is the name we want to give to this output video

53

and we're going to call it.

54

Well very simply outputs that and before this way I'll put that image for a second argument which is

55

actually the frequence how many frames per second do we want.

56

And so while the name of the argument is yes you have to specify it.

57

And this is equal to this.

58

FP is variable here that we got things to get made a data function from our reader object.

59

So FP as equals Appius are right.

60

So now we have everything we have.

61

All we need to start this for loop and process each of the images of the funny the video.

62

And so of course you understand that in each step of the loop we're going to work on a specific frame

63

of the video and on that frame we're going to apply the detect function to detect the objects in the

64

frame and print the rectangles.

65

All right.

66

So let's start this for loop for I.

67

So I will just correspond to the number of the image that is processed.

68

So you know I'm going to go from 0 to I told you there's going to be 68 fremd so I'm going to go from

69

zero to 68 or 67 something like that.

70

So for I and then frame of course we're iterating over the frames of the video.

71

That's why I'm taking frame that's just the name of the variable that will exactly correspond to each

72

of the frames of the video.

73

So I-frame in.

74

And then we can use and enumerate parenthesis reader that that will just iterate through all the frames

75

of the reader video.

76

The funny the video.

77

So for I-frame numerate reader.

78

Well what do we do.

79

We simply need to apply to detect method on this frame right here to have some objects detected by the

80

net which is or as is the neural network that we created associated to the right transformation to transform

81

object here to make sure that this frame can be accepted into this net.

82

All right.

83

So nothing more easy to do here.

84

We apply the detect function to our frame with our neural network net and with our transformation transform.

85

However there is just little trick here.

86

NET is actually an advanced structure.

87

Remember it's an object of the build as the class.

88

And in order to get this neural network that is expected by the Dodik function it's not only that the

89

22 input it's actually not that level that just to align with the way to build as is the function was

90

made but basically net that yvel represents our new network net from which we get the output y and therefore

91

the detections on each frame.

92

So perfect we detected the objects on our frame but remember that this detect function returns actually

93

the frame the processed frame with the detected object and therefore I'm going to introduce here a new

94

variable that will represent that process frame.

95

And there is actually no danger to call it again frame so I'm just overwriting the frame here.

96

But that's totally OK here.

97

The frame is the original frame with no detection made yet and this frame is the new frame.

98

After we apply the detect function with the detector rectangles.

99

All right.

100

So now the loop is now over.

101

What do we need to do.

102

Well each time we get a new process frame with the objects detected we need to append this frame to

103

our writer output video and that's exactly what we're going to do now to append a frame to our right

104

of video.

105

We simply need to take our writer object than dot and then we use append underscore data function to

106

which we need two input.

107

Of course what we want to append to the writer output video and that is of course this new preset frame

108

with the detected object.

109

Perfect.

110

So now the process for them is appended.

111

And now we can just add print I just to see during the detection which frame we reached.

112

All right.

113

That's just a practical thing to see the number of the process frame will be displayed during the detection.

114

And finally last line of code.

115

Well we just need to close the process that manages the creation of this video and to close it.

116

We just need to take our writer and then add that and then close parenthesis the close function that

117

will close the process and we'll get the output video in that same repertory.

118

That is our working directory folder.

119

All right so that's it.

120

We're ready to watch the final output.

121

So what do you thing do we do it in Statoil.

122

Well yeah let's do it let's do it right now so we simply need to select all the code and execute.

123

There we go.

124

No error just a warning that's OK.

125

That's just a warning for f MPEG library but it's OK.

126

And here we can see the number of the frame that is processed.

127

You can see that it's going actually pretty fast.

128

So we'll get the final output video.

129

Very quickly I told you there's about sixty eight frames to be processed on each frame.

130

Replying the direct function to the object.

131

Let's see what happens and we'll get quickly to the final result.

132

All right so it's about to end very very soon.

133

Yeah.

134

OK.

135

So it went from zero to 67 so there was indeed sixty eight frames to process that is to detect some

136

object in the video in a two seconds video.

137

OK.

138

So let's watch the final output.

139

But before that I just want to show you again the original video funny Doug before.

140

So let's watch this again.

141

BOING BOING BOING BOING BOING BOING.

142

All right the duck bouncing on the field.

143

And now let's see what our mole was able to do.

144

So there is this dog here a human here Carol here and some other humans here.

145

Let's see what this mole was able to detect.

146

Going to close that video.

147

I'm going to get my outputs and let's watch the result.

148

Ready.

149

And play all right amazing job Doug was detected the human was detected and I didn't have time to see

150

how well your role was detected but I think I saw some detections on this humans.

151

Let's watch this again.

152

That's an amazing job you can try to do that with open Sivi or some other models.

153

I'm not sure you get such a great result.

154

I actually tried it with open city and I definitely didn't get the same results.

155

There were rectangles everywhere.

156

So that didn't work.

157

But with this SSD model the detection is amazing the drug is perfectly well detected.

158

So now let's see let's see for the other detection.

159

So the human is also detected this human here but it's quite big in the video.

160

So of course it's detected.

161

The drug is well detected as well.

162

Let's see some other OK.

163

So the humans behind are hidden by the arms of course they're not yet detected but let's see what happens

164

next.

165

All right that's what I'm talking about here on this special frame.

166

This special frame is very interesting because not only we can see the humans behind detect it very

167

well detected it's actually detected this lady here and and also Kiril was detected but we lost the

168

detection on the dog.

169

And why is that.

170

It's because the dog merged with Curiel you see CULE has the upper body of Kiril but the lower body

171

of the dog and therefore the model things that one same person.

172

And that's why it detected the person.

173

So that's pretty funny.

174

And then if I move onto the next frame Well the detection of the real person is gone.

175

And we got back the detection of the dog.

176

That's a pretty funny thing that happened here.

177

OK.

178

And then all right we had some more dog and more Duguay.

179

So that's pretty cool isn't it.

180

The dog is well detected the humans will detect it and sometimes we get some other detections on other

181

humans that are much harder to detect.

182

So I hope you are convinced by the power of this as demurral you can actually try with the other ones

183

you'll see that it's a pretty great job that was done here by the SS The neural network in the next

184

tutorial.

185

I'll give you a little homework.

186

It will be to do some detection and some other video some very cool and really really beautiful video

187

of some horses running on some field and filmed by the drone.

188

I'd like you to try this because I like you to keep in mind that this model can not only detect common

189

things like humans and dogs but also many other objects like horses boats cars planes whatever.

190

I think between 30 to 40 objects so that will be a funny homework to do.

191

Not difficult.

192

But don't worry we'll get back to difficult things in module three with deep convolutional Ganns.

193

So I'll see you in the homework and module 3.

194

And until then enjoy can do revision.