Skip to content

http: merge chunks during chunked encoding - #47630

Closed
gjgmenendez wants to merge 4 commits into
nodejs:mainfrom
gjgmenendez:issue#57
Closed

gjgmenendez wants to merge 4 commits into
nodejs:mainfrom
gjgmenendez:issue#57

Conversation

@gjgmenendez

@gjgmenendez gjgmenendez commented Apr 19, 2023 •

Copy link
Copy Markdown

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/http
  • @nodejs/net

@nodejs-github-bot nodejs-github-bot added http Issues and PRs related to the http subsystem. needs-ci PRs that need a full CI run. labels Apr 19, 2023
@mscdex

mscdex commented Apr 20, 2023

Copy link
Copy Markdown
Contributor

What is the context for this?

@bnoordhuis

Copy link
Copy Markdown
Member

@tniessen tniessen changed the title Issue#57 http: merge chunks during chunked encoding Apr 20, 2023
@tniessen tniessen added the performance Issues and PRs related to the performance of Node.js. label Apr 20, 2023

@ronag ronag left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs a little more work. 🤗

Comment thread lib/_http_outgoing.js
);
}

let chunkBuffer = Buffer.alloc(0);

@ronag ronag Apr 21, 2023 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having this as a global variable will cause concurrent requests to interleave and become corrupt. The buffer needs to be scoped to the response object itself.

Comment thread lib/_http_outgoing.js
msg._send(crlf_buf, null, null);
msg._send(chunk, encoding, null, len);
ret = msg._send(crlf_buf, null, callback);
chunkBuffer = Buffer.concat([chunkBuffer, Buffer.from(chunk)]);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concatenating buffers is very slow. They require a memory allocation + memory copy. Just keep an Array of buffers.

Comment thread lib/_http_outgoing.js
Comment on lines +1047 to +1053
if (!this._headerSent && Buffer.byteLength(chunkBuffer)) {
this._send(NumberPrototypeToString(Buffer.byteLength(chunkBuffer), 16), 'latin1', null);
this._send(crlf_buf, null, null);
this._send(chunkBuffer, null, null);
this._send(crlf_buf, null, callback);
chunkBuffer = Buffer.alloc(0)
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be done in nextTick (i.e., connectionCorkNT) otherwise, there is no point to all of this... the number of buffered chunks would always be 1.

Comment thread lib/_http_outgoing.js
this.socket.uncork();
}
this[kCorked] = 0;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unnecessary change

})
);

server.listen(3000, () => {});

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't test anything related to the problem we are trying to solve.

@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been marked as stale due to 90 days of inactivity.
It will be automatically closed in 30 days if no further activity occurs. If this is still relevant, please leave a comment or update it to keep it open.

@github-actions github-actions Bot added the stale Issues and PRs marked stale due to inactivity and scheduled for automatic closure. label Jul 28, 2026
@avivkeller avivkeller closed this Aug 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

http Issues and PRs related to the http subsystem. needs-ci PRs that need a full CI run. performance Issues and PRs related to the performance of Node.js. stale Issues and PRs marked stale due to inactivity and scheduled for automatic closure.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants