Repository navigation
IPC deferred with larger payloads (macOS only) #30009
Description
- Version: v12.12.0
- Platform: macOS v10.14
- Subsystem: child_process
When creating a child process with fork() and IPC messages sent inside a loop are received ~immediately by the parent when the data is 'small' (<8k?), but when the data is 'large' (>10k?) on macOS the parent process does not receive the messages until the loop has finished. On Windows and Linux no such problem occurs.
Repro steps:
- Run
node server.jsusing code below on macOS. curl http://localhost:8080/15- You will see alternating send/receive messages (correct)
curl http://localhost:8080/123456
EXPECTED: Alternating send/receive messages in the console with interleaved timestamps.
ACTUAL: All send messages occur first, followed by all receive messages.
server.js
const opts = {stdio:['inherit', 'inherit', 'inherit', 'ipc']};
const child = require('child_process').fork('test-child.js', [], opts);
let lastUpdate = {};
child.on('message', msg => console.log(`parent: receive() ${msg.data.length} bytes`, Date.now()));
require('http').createServer((req, res) => {
console.log(req.url);
const match = /\d+/.exec(req.url);
if (match) {
child.send(match[0]*1);
res.writeHead(200, {'Content-Type':'text/plain'});
res.end(`Sending packets of size ${match[0]}`);
} else {
res.writeHead(404, {'Content-Type':'text/plain'});
res.end('what?');
}
}).listen(8080);worker.js
if (process.send) process.on('message', msg => run(msg));
function run(messageSize) {
const msg = new Array(messageSize+1).join('x');
let lastUpdate = Date.now();
for (let i=0; i<1e7; ++i) {
const now = Date.now();
if ((now-lastUpdate)>200) {
console.log(`worker: send() > ${messageSize} bytes`, now);
process.send({action:'update', data:msg});
lastUpdate = Date.now();
}
Math.sqrt(Math.random());
}
console.log('worker done');
}Output of a run:
/15
worker: send() > 15 bytes 1571324249029
parent: receive() 15 bytes 1571324249034
worker: send() > 15 bytes 1571324249235
parent: receive() 15 bytes 1571324249235
worker: send() > 15 bytes 1571324249436
parent: receive() 15 bytes 1571324249436
worker done
/123456
worker: send() > 123456 bytes 1571324276973
worker: send() > 123456 bytes 1571324277174
worker: send() > 123456 bytes 1571324277375
worker done
parent: receive() 123456 bytes 1571324277391
parent: receive() 123456 bytes 1571324277391
parent: receive() 123456 bytes 1571324277393
Activity
bnoordhuis commentedon Oct 18, 2019
Can you explain why that's your expectation? Keep in mind that the behavior you observed on other platforms is not at all guaranteed behavior.
Phrogz commentedon Oct 18, 2019
Given 6 scenarios (Win/Linux/macOS × small/large payloads), when five out of the six behave the same I think it is reasonable to expect that the sole remaining case would behave in a consistent manner.
The purpose of
send()with IPC is to communicate a message between processes. If you told me thatsend()might queue up every message until you chose to quit the programs involved, I would say thatsend()was useless. Thankfully, it does not behave this way. It behaves as one would hope: it ~immediately sends the message to be received by the other side. Hooray!Except that in this one case on one platform it behaves differently.
It is hard to keep that in mind when I did not previously know it. I would appreciate links to sources that elaborate on this claim. Do you simply mean that the documentation on
child_process.send()provides no guarantees or discussion on the timing of delivery?bnoordhuis commentedon Oct 19, 2019
That's right and that's intentional.
Apropos the documentation, this paragraph is relevant to your test case:
targos commentedon Dec 26, 2020
This seems to have been answered.