Skip to content
This repository was archived by the owner on Jun 11, 2026. It is now read-only.
This repository was archived by the owner on Jun 11, 2026. It is now read-only.

Consider upgrading to simdjson 0.4.0 #17

Open

Description

@lemire

Version 0.4.0 is available

Highlights

  • Test coverage has been greatly improved and we have resolved many static-analysis warnings on different systems.

New features:

  • We added a fast (8GB/s) minifier that works directly on JSON strings.
  • We added fast (10GB/s) UTF-8 validator that works directly on strings (any strings, including non-JSON).
  • The array and object elements have a constant-time size() method.

Performance:

  • Performance improvements to the API (type(), get<>()).
  • The parse_many function (ndjson) has been entirely reworked. It now uses a single secondary thread instead of several new threads.
  • We have introduced a faster UTF-8 validation algorithm (lookup3) for all kernels (ARM, x64 SSE, x64 AVX).

System support:

  • C++11 support for older compilers and systems.
  • FreeBSD support (and tests).
  • We support the clang front-end compiler (clangcl) under Visual Studio.
  • It is now possible to target ARM platforms under Visual Studio.
  • The simdjson library will never abort or print to standard output/error.

Version 0.3 of simdjson is now available

Highlights

  • Multi-Document Parsing: Read a bundle of JSON documents (ndjson) 2-4x faster than doing it individually. API docs / Design Details
  • Simplified API: The API has been completely revamped for ease of use, including a new JSON navigation API and fluent support for error code and exception styles of error handling with a single API. Docs
  • Exact Float Parsing: Now simdjson parses floats flawlessly without any performance loss (accurate number parsing simdjson/simdjson#558).
    Blog Post
  • Even Faster: The fastest parser got faster! With a shiny new UTF-8 validator
    and meticulously refactored SIMD core, simdjson 0.3 is 15% faster than before, running at 2.5 GB/s (where 0.2 ran at 2.2 GB/s).

Minor Highlights

  • Fallback implementation: simdjson now has a non-SIMD fallback implementation, and can run even on very old 64-bit machines.
  • Automatic allocation: as part of API simplification, the parser no longer has to be preallocated-it will adjust automatically when it encounters larger files.
  • Runtime selection API: We've exposed simdjson's runtime CPU detection and implementation selection as an API, so you can tell what implementation we detected and test with other implementations.
  • Error handling your way: Whether you use exceptions or check error codes, simdjson lets you handle errors in your style. APIs that can fail return simdjson_result, letting you check the error code before using the result. But if you are more comfortable with exceptions, skip the error code and cast straight to T, and exceptions will be thrown automatically if an error happens. Use the same API either way!
  • Error chaining: We also worked to keep non-exception error-handling short and sweet. Instead of having to check the error code after every single operation, now you can chain JSON navigation calls like looking up an object field or array element, or casting to a string, so that you only have to check the error code once at the very end.

Activity

  1. changed the title [-]Consider upgrading to simdjson 0.3.0[/-] [+]Consider upgrading to simdjson 0.3.1[/+] on Apr 2, 2020
  2. dongx-db commented on May 14, 2020

    @dongx-db
    Collaborator

    Thanks for bringing this up. I will start working on it once I get some free cycles.

  3. changed the title [-]Consider upgrading to simdjson 0.3.1[/-] [+]Consider upgrading to simdjson 0.4.0[/+] on Jun 24, 2020
  4. lemire commented on Jun 24, 2020

    @lemire
    Author

    Version 0.4.0 is available.

  5. dongx-db commented on Mar 5, 2021

    @dongx-db
    Collaborator

    Working on it. This time will pull it all the way up to the latest version.

  6. lemire commented on Mar 5, 2021

    @lemire
    Author

    @dongx-psu We are close to releasing 0.9.

    You may want to have a look at our On Demand API which is going to become our primary front-end.

    ping John Keiser (@jkeiser)

  7. dongx-db commented on Mar 18, 2021

    @dongx-db
    Collaborator

    Daniel Lemire (@lemire) I didn't see the on-demand API documentation? Could you kindly refer to me where it is?

    Also, for padding, I have two questions: (1) what will happen if I don't pad the buffer. (2) If I have a buffer sent having enough memory. During the parsing, will it parse beyond the length I specify? Will it change anything on the data beyond the length I specify?

  8. lemire commented on Mar 19, 2021

    @lemire
    Author

    @dongx-psu As of 0.9 (release yesterday), all our documentation focuses on On Demand by default. The DOM documentation is hidden away.

    Will it change anything on the data (...)

    The simdjson library will never touch the inputs since you pass a pointer to constant memory.

    During the parsing, will it parse beyond the length I specify?

    We give ourselves the permission to read 32 bytes (SIMDJSON_PADDING) beyond the end of the provided input. You can put anything in these extra 32 bytes (garbage).

  9. dongx-db commented on Mar 19, 2021

    @dongx-db
    Collaborator

    Daniel Lemire (@lemire) So I don't see parse_many as an API for on-demand? So, the iterate function will iterate through all the documents? And also, will things like current_index work for that API as well?

  10. lemire commented on Mar 19, 2021

    @lemire
    Author

    @dongx-psu Currently, parse_many is strictly a DOM API but we have plans to bring it over to On Demand.

  11. dongx-db commented on Mar 19, 2021

    @dongx-db
    Collaborator

    Thx for letting me know. I will work on the DOM API for now then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions