All language subtitles for 009 Streams in Practice_Downloadly.ir_en

All right, now lets actually work with streams.

And again, starting by creating a new file.

All right. Now lets say that for some reason in

our application, we need to read a large text file from

the file system, and then send it to the client.

So how would we do that?

Well there are multiple ways and we're going to explore

a few of them starting with the most basic one

and moving all the way to the best way of doing this.

Okay, so the first thing is to require the file system

package just like before, and also the HTTP module.

Let me now show you a nice trick,

so lets actually create a server like this.

So, require the HTTP package, and then from here we can

right away call createServer. So just like this.

Okay, so the result of requiring HTTP, is the

HTTP object and on there we can then use createServer.

Just like this, and then save it into a variable.

So that is yet another way of creating a new server

with writing even a bit less code.

Okay and so just like before lets now list to the request

event, and specify our callback here.

So, the first solution that we're going to use is the

easiest and most straight-forward one.

Which is to simply read the file into a variable,

and then once that's done, send it to the client

in the way that we already know how to do it.

So that one's very simple, and let me write a comment

here for that. So solution one, and so

fs.readFile then our test file again.

And then once that's ready, we call our callback function

in which we have access to the data.

But first lets handle the error.

So for example in case that the file does not exist,

and in that case we simply log the error to the console.

But otherwise, we simply send this data back to the client.

So we use the response object just like we did before

many times, and then .end and the data.

So, give it a save, and so that is the first

and the simplest solution, right?

But before we can test this, we need to actually also

start a sever, right? So we use listen to do that.

And the port, the same one as before, then localhost

and our callback function.

Okay, so lets see if this actually works.

And of course not, because I actually didn't even start

the application.

So now it says listening, and now lets see

what happens here.

And here we go. So this file is huge, it has

Node.js is the best written in it like 10000 times

or something, and so it takes a lot of time

until it loads entirely. And we're not really interested in

loading everything, so lets stop this here,

and go back to our code. So this works just fine, the

solution one that we have here in this case,

but the problem is that with this solution,

node will actually have to load the entire file

into memory, because only after that's ready,

it can then send that data. Now this is a problem

when the file is big, and also when there are a ton

of requests hitting your server. Because the node process

will very quickly run out of resources and your app will

quit working, everything will crash, and your users

will not be happy believe me. So this solution here

does work when we're just creating something small

locally for just ourselves, for example.

But in a production-ready application, you cannot use

a piece of code like this. Okay? So lets move on to

our second solution. And in that solution,

we will actually use streams.

Okay, so lets comment out this part,

and move on to solution number two.

And the idea here is that we actually don't really

need to read this data from the file into a variable, right?

We don't need this variable. So instead of reading

the data into a variable, and having to store that variable

into memory, we will just create a readable stream.

Then as we receive each chunk of data,

we send it to the client as a response

which is a writable stream remember.

So let me now show you how we can use streams.

So we create a variable, lets call it readable here,

and then from the file system again, we use createReadStream

Then this of course small. And now the name of the file

that we're trying to read. So that is again, test-file.txt.

Okay, perfect. So this now creates a stream from

the data that is in this text file, which we can then

consume piece by piece. So chunk by chunk.

And how do we do that? Well, remember from the last

lecture, that each time there is a new piece of data

that we can consume, a readable stream emits the

data event. Okay, and so we can listen to that,

just like we learned in the events lecture.

So readable.on, data, and then our callback function.

And in our callback function, we have access to

that piece of data, so to that chunk.

Let me call it chunk here in our callback function

and so now we can handle this piece of data.

And what we're going to do with this piece of data,

with this chunk, is to actually write it to a

writable stream, which is the response.

So, res.write, this chunk. Okay?

So again remember that this response is a

writable stream. So just as I mentioned in the previous

video, right? And so we can now use the write method

to send every single piece of data into that stream.

Okay, and with this effectively we're streaming

the content from the file right to the client.

Okay, you understand the difference?

So before, we write everything at once into a variable,

and once that was ready, we then sent that entire piece

of data back to the client. But in this situation,

with the stream, it is different.

We're effectively streaming the file, so we read one piece

of the file, and as soon as that's available,

we send it right to the client, using the write method

of the respond stream. Then when the next piece is available

then that piece will be sent, and all the way until

the entire file is read and streamed to the client.

Okay, so now just to finish, we also have to handle

the event when all the data is read.

So when the stream is basically finished reading the data

from the file.

So on that case, the end event will be emitted,

and so as soon as that happens what we're going to do

is to do res.end, okay? And we used this one before,

so calling end on the response, we did that before, right?

And now, it makes actually more sense, because again,

the response is also a stream, and the end method

signals that no more data will be written to this

writable stream, okay?

So up here all we did was actually use res.end with the

data in it. So we did no streaming, all we did was

in the end to send some data.

Now in this case, we're not passing anything

into this end method, because we already sent all the data

using res.write, chunk by chunk,

and so when the readable stream is finished reading

its file, well all we have to do is to then signal

that we're ready using res.end like this, okay?

So we always need to use this data and this end event here

one after another like this. Because otherwise,

the response will actually never really be sent

to the client. Okay, so without this piece here,

this entire solution would not really work, okay?

So lets now close this one, start again,

and read again, and so you see that its actually working

again, okay? Now this time without the problems we had

with the first solution. Lets stop this here, and go back

to our code, because I want to show you now,

that there is another event that we can listen to

on a readable stream, which is the error event.

So readable.on('error') and in this callback function

we have access to the error object.

Okay so in this case we will log this error to the console,

and then send the result of file not found.

And we can also set the status code to an error,

so 500 for example. So usually it's automatically set to 200

which means okay, but in this case we have a server error,

which means that we want to send back the code 500.

All right, so lets now...

Actually I need to quit this again.

So lets see what happens now.

Oh we already have an error right here,

res.status is not a function. And yeah,

actually its statusCode. So this is the way that

you write it in express, so I'm already used to writing

express so much, so express is a node.js framework

which we're going to use in the rest of the course

and in express you do it like this, and so that was

the problem. So I'm already a bit too used

to writing express.

So lets go back, and yeah so now we see file not found,

and if we open up the dev tools, you see our 500 error code

that we just sent before, and if we go to our network tab,

lets try that again, you have the status code here as well.

So just like we saw in one of the previous videos in another

section actually. So this is how we can inspect this

kind of stuff in the network tab.

All right, so lets fix it here, give it a save,

and okay so this again works just perfectly

but there still is a problem with this approach

that I just showed you. And the problem is that our

readable stream, so the one that we're using to read

the file from the disk, is much much faster than actually

sending the result with the response writable stream

over the network. And this will overwhelm the response

stream, which cannot handle all this incoming data so fast.

And this problem is called backpressure.

And it's a real problem that can happen in real situations.

So in this case, backpressure happens when the response

cannot send the data nearly as fast as it is receiving it

from the file, does that make sense?

So we have to fix that solution,

and come up with an even better one.

And so, we will create solution three,

and that is actually the final one,

Okay? So no more solutions than three.

So the secret here is to actually use that pipe operator

that I mentioned in the last video, okay?

So the pipe operator is available on all readable streams,

and it allows us to pipe the output of a readable stream

right into the input of a writable stream, okay?

And that will then fix the problem of backpressure

because it will automatically handle the speed basically

of the data coming in, and the speed of the data going out.

Okay, so lets actually get our readable stream here.

Okay, so that is our readable stream, and now

all we have to do is to take our readable stream,

use the pipe method on it, and then put in a writable stream

and that is the response. And that's actually it.

That's all we have to do for this solution, okay?

So it always works like this, let me write it out here

as a comment. So we need a readable source basically, okay

so again this is just to explain it to you,

then we can use the pipe on it, and here we will have to put

a writable destination. So this is where our data

comes from, and it has to be a readable stream,

and that data we can then pipe into a writable destination.

So in this case our destination is the response, okay?

Now this stream here can actually be a duplex or a

transform stream as well, but what matters is that

we can write to the stream. And response of course,

is such a stream. So we can of course write to the response

that will be sent to the client, okay?

So the pipe operator automatically solves the problem

of backpressure that we had previously.

And it's also actually a much more elegant and

straight-forward solution. So what we did here before

was just to show you all the ways in which we can

use the stream methods and events in order to create this

kind of solution and of course they have many use cases,

but in a problem like this, the pipe operator is actually

the best solution. Behind the scenes it does something like

this here actually, but again in a

much more straight-forward way for us to write,

because it handles all the stuff internally

behind the scenes. So I believe that pipe here is actually

the easiest way of consuming and writing streams, unless

of course, as I mentioned, we need a more customized

solutions. And then, we have to use these more complicated

tools like these events and methods that

I showed you before. All right, so just to finish,

lets quit the process here, start it again,

and of course, see if it still works, which it does.

And so, our work is done here. So we're finished with this

lecture, and ready to go straight to the next one.