Repository navigation
Lost input from child processes due to initially-flowing net.Socket. #5034
Description
When opening child processes with child_process.spawn using 'pipe' for its stdout or stderr, the resulting stream is already in the flowing state when it's returned, and some data can be lost when consuming the stream using the 'readable' event + read(). A minimal example:
const spawn = require('child_process').spawn;
const ls = spawn('echo', ['123']);
console.log("flowing:", ls.stdout._readableState.flowing);
ls.stdout.on('readable', function() {
console.log("read:", ls.stdout.read());
});On recent node versions, no data is read; the 'readable' event only fires once, for end-of-stream. On older versions, the first chunk of data is read. I'm not sure which exact version introduced the bug, but:
% node --version && node /tmp/test.js
v0.10.36
flowing: false
read: null
read: \<Buffer 31 32 33 0a\>
read: null
% node --version && node /tmp/test.js
v5.1.0
flowing: null
read: null
% node --version && node /tmp/test.js
v5.5.0
flowing: null
read: nullPotentially related to #445. Seems to be the root cause of a few issues affecting GHCJS:
- ghcjs + node - readProcess (or any runInteractiveProcess) fails to read from stdout ghcjs/ghcjs#453
- quick install ghcjs fail : fatal:program ghcjs is required but could not be found ghcjs/ghcjs#436
- ghcjs could not be determined luigy/try-stack-reflex#5
- Can't setup ghcjs with stack: ghcjs version could not be determined. commercialhaskell/stack#1496
- stack setup ghcjs fails commercialhaskell/stack#1437
- Building with GHCJS intermittently crashes commercialhaskell/stack#1629
The interaction between https://github.com/nodejs/node/blob/master/lib/internal/child_process.js#L228 and https://github.com/nodejs/node/blob/master/lib/net.js#L165 looks suspicious to me, but I don't know enough about nodejs' internals to suggest an appropriate fix.
Activity
awpr commentedon Feb 2, 2016
Also relevant, replacing the 'echo' with a read of a named pipe shows different behavior based on how quickly input becomes available on the pipe:
const spawn = require('child_process').spawn;
const ls = spawn('cat', ['/tmp/the_named_fifo']);
% cat /tmp/test.js
console.log("flowing:", ls.stdout._readableState.flowing);
ls.stdout.on('readable', function() {
console.log("read:", ls.stdout.read());
});
% mkfifo /tmp/the_named_fifo
% node --version && node /tmp/test.js & echo hi >> /tmp/the_named_fifo; wait
v5.1.0
[1] 13721
flowing: null
read: null
[1] + done node /tmp/test.js
% node --version && node /tmp/test.js & sleep 1; echo hi >> /tmp/the_named_fifo; wait
v5.1.0
[1] 13730
flowing: null
read: <Buffer 68 69 0a>
read: null
[1] + done node /tmp/test.js
mscdex commentedon Feb 2, 2016
I think I have a fix for this coming up ...
mscdex commentedon Feb 2, 2016
#5036 should fix this
child_process: fix data loss with readable event
child_process: fix data loss with readable event
child_process: fix data loss with readable event
child_process: fix data loss with readable event
child_process: fix data loss with readable event
child_process: fix data loss with readable event
child_process: fix data loss with readable event
child_process: fix data loss with readable event
child_process: fix data loss with readable event