Repository navigation
Stream 'readable' event oddness #198
Description
- Version: 6.2.2
- Platform: Windows 7 x64
The 'readable' event is emitted when there is data available to be read from the stream... The 'readable' event will also be emitted once the end of the stream data has been reached but before the 'end' event is emitted.
Why in this script (saved in UTF-8 with BOM) the handler is called no more and no less than twice?
/******************************************************************************/
'use strict';
/******************************************************************************/
const fs = require('fs');
const readStream = fs.createReadStream(__filename, { encoding: 'utf8' });
readStream.on('readable', () => {
console.log(readStream.read(3));
});The output:
/*
***
Activity
vsemozhetbyt commentedon Jun 20, 2016
Call stacks for both event firing. Is the second emitting taken into account by the documentation?
addaleax commentedon Jul 13, 2016
I think the
end of stream datapart ofThe 'readable' event will also be emitted once the end of the stream data has been reached but before the 'end' event is emitted.line refers to theonEofChunk, so yeah, I think the docs are correct.I don’t know the specific reasons for this, though.
vsemozhetbyt commentedon Jul 14, 2016
@addaleax Does
onEofChunkmeansend of ChunkorEOF Chunk? If the first, is end of chunk (in the function) == end of the stream data (in the doc)? It seems not obvious. If the second, why does it fire if I does not read all the data, only some bytes? Is data read to the end anyway on the exit?addaleax commentedon Jul 14, 2016
onEofChunkmeansEOF, i.e. after all data from the source has been read into theReadable.I’m not entirely sure I understand the question, but maybe the source of confusion is the fact that
Readablestreams actually buffer data to a certain degree, to have more data available for reading when the stream becomes readable.The size of that buffer is configurable through the
highWaterMarkoption, and if you set that low enough, the stream will not read then entire file:Here the
'readable'listener only fires once, because only 5 bytes are read from the source into the stream’s internal buffer, of which only 3 bytes are used. After the.read()call, there will be 2 more bytes in the streams internal buffer; the stream will remain paused until these have been extracted using.read(), too (e.g. try setting the3to5and see what happens).vsemozhetbyt commentedon Jul 14, 2016
O, I think I see now. So
once the end of the stream data has been reachedalso meansincluding being exhausted by internal buffer?addaleax commentedon Jul 14, 2016
It is rather fired “as soon as it is clear that no more data will be added to the internal buffer, because the data source (i.e. the file itself) contains no more content”. After the last
readableevent (the onEofChunk one), the user can call.read()a few times to get the rest of the buffered data.The event that is fired when all data has been read and the internal buffer has fully been cleared is the
endevent.vsemozhetbyt commentedon Jul 14, 2016
Phew) Got it. Thank you for all your time and explanations!
addaleax commentedon Jul 14, 2016
Glad if it helped, and of course feel free to ask more questions here!
saharshjain commentedon Mar 6, 2020
@addaleax I have a csv file and it contains 6K records and I am using streams to read and csv-parse package only.
I could see the readable event getting fired after few chunks of rows. Shall I understand that as soon as it starts reading the chunk and no more could be buffered it emits 'readable' event and when it start reading next chunk and end of chunk is reached it is fired again and so on.
So practically