Repository navigation
BigInt should be handled by native codec instead of extension codec #115
Description
It is not a good choice to handle 64bit integers with extension codec. While 64 bit integers are built-in types of msgpack and BigInt is a built-in type of javascript, solving 64 bit problem with extension codec is not an compatible way when making cross-language systems. If javascript developers use an extension codec to support 64bit integers, then developers in other languages will have to write an corresponding extension codec, while msgpack implementation in other languages (c++, c, python, java, etc.) natively support mapping 64 bit integers into their language-specific type.
To support BigInt, I think this library should map 64 bit integers to BigInt natively or this library should add support to register custom codec for built-in types.
original issue #114
Activity
gfx commentedon Jun 23, 2020
BigInt is not compatible with numbers (i.e. 64-bit floating-point numbers) in JavaScript, and BigInt is not compatible with 64-bit ints in general, so I suppose implicit mapping of BigInt and 64-bit ints should cause problems.
However, if one knows what they are doing, its mapping could be helpful for cross-platform cooperation, so I'd like to consider an optional feature to map BigInt and 64-bit ints.
bin-y commentedon Jun 23, 2020
Glad to hear that you have take mapping BigInt and 64-bit integers into consideration. But I don't think it is optional. Yes, BigInt is not compatile with javascript Number as Number in javascript is something like
double, but no, BigInt should be compatible with 64-bit integers of msgpack as integers and floats are different things in general and as the existence of BigInt.asIntN and BigInt64Array.shchemelevev commentedon Sep 10, 2020
+1
joshyrobot commentedon Jun 11, 2021
What? Of course it is, we even have wonderful methods on
DataViews like.getBigUint64(). This is the only proper way to support 64 bit integers, any other method is going to be far less compatible with plain numbers.arendjr commentedon Apr 20, 2022
I also believe adding native
BigIntsupport to handle MessagePack's native 64 bit integers is the right way to go. I also see @gfx's concern, because MessagePack tends to pick the smallest representation when encoding numbers, which could lead to an annoying schism on the JS side.For example, if I have an
u64in Rust that should be encoded with MessagePack, MessagePack will happily encode it as a 7-bit integer if the value fits, which would come out as anumberon the JS side. But if the value were larger than2^32 - 1it would use the 64-bit encoding, which would suddenly come out as aBigInton the JS side. This can be annoying if you have to deal with both (incompatible) types on the JS side.The current "solution" at least supports number up to
2^53 - 1consistently on the JS side, but it's not a solution at all for anyone who needs integers bigger than that. So I agree having an optional feature to opt-in to the use ofBigIntwould be the right way to go that I believe would satisfy everyone.jasonpaulos commentedon May 5, 2022
I've just made #211, which is a PR to add BigInt support in an opt-in and flexible way. Feedback/support would be appreciated.
gfx commentedon Mar 6, 2023
Thank you for your effort, @jasonpaulos, but I've made another PR for bigint support: #223 , which introduces an option
useBigInt64. It's not so much flexible. It introduces a new mapping for JavaScript's bigint and MessagePack's int64/uint64, as JavaScript's DataView does (it uses DataView's native features, indeed).I'll release it at the end of March 2023 if nobody has objections.
jasonpaulos commentedon Apr 13, 2023
I responded in that PR with an issue I noticed: #223 (comment)
We are currently using an old fork of this library with bigint support. I would very much like to stop using that fork and use the original library again, but the issue I pointed out in my comment would prevent us from migrating.
WebReflection commentedon Feb 21, 2025
awkwardly enough, the 2^53-1 limit of number kind in JS cannot really be represented elsewhere with a meaning ... it's either int32, uint32, or int64 and uint64 ... in here it looks like the implementation propagates JS numbers convention elsewhere, but there I say it could be disambiguate in way that is sompatible with other platforms/PLs too.
Stick with
int32oruint32type (same goes for float) and promote to bigint anything above that so that other PLs can just use x64 variant in there? eventually we'll have issues in decoding on the JS side, but imho bigint is super rare out there and if anyone needs that back explicitly, it could always doBigInt(String(number))to have it back?