Repository navigation
performance of encodings (hex, base64, base64url) #128
Description
In the last few days I was investigating the performance of hex and especially base64 and base64url
Added benchmarks nodejs/node#50348
base64 encoding is using the functionality from the base64 dependency.
base64 decoding is not using the functionality from the base64 dependency. We have a custom implementation, which handles the base64 decoding gracefully. So a whitespace does not result in an error but gets ignored.
base64url encoding is a custom implementation. So it is slower than it could be.
base64url decoding is a custom implementation. So it is slower than it could be.
hex encoding is a custom implementation. So it is slower than it could be.
hex decoding is a custom implementation. So it is slower than it could be.
Maybe this is something to be implemented in simdutf?
Activity
lemire commentedon Oct 23, 2023
All good ideas/pointers.
It would be possible to design a base64 library specifically for the needs of Node.js. We could throw in base16 (hex) and so forth. Handling spaces efficiently is possible.
lemire commentedon Dec 13, 2023
The base64 decoder is robust with respect to spaces but it seems to ignore any non-base64 character, actually...
See what the specification says...
But look...
The hex decoder seems to stop on the first non-hex character:
This seems documented:
aduh95 commentedon Dec 13, 2023
What specification? AFAIK
Bufferis a Node.js API, the only "specification" would be Node.js docs.lemire commentedon Dec 14, 2023
Yes. I quoted the documentation.
lemire commentedon Mar 16, 2024
Base64 support is coming soon in simdutf: simdutf/simdutf#375
lemire commentedon Apr 8, 2024
atob performance has been greatly improved by @anonrig
nodejs/node#52381
So this handles part of the issue.
lemire commentedon Apr 8, 2024
@anonrig is handling part of the rest of the issue in nodejs/node#52428