[igorls]

Summary

Moves WormDB to Zig 0.17.0. It also adds a performance pass over the server, storage, vector and proof paths, several correctness fixes found along the way, a release workflow, and benchmark tooling.

Zig 0.17

• The codebase, build.zig and CI now use Zig 0.17.0. Std API shims stay centralized in src/core/compat.zig.
• deps/meshguard is bumped to its Zig 0.17 release.

Fixes

• cluster: one-way replication streams no longer send acks nobody reads. vrabitq_install is handled in replication, with a test.
• uring: partial sends track the offset and are resubmitted.
• gateway: sendWireResponseFrame supports the 64-bit WebSocket length header.
• compat (Windows): the reusable AFD event handle lives on Stream. NO_FAST_IO is kept on the socket path.
• storage: scanPrefixCut no longer reserves memory sized from a client-supplied limit (capped at 1024), with a test.
• vector: insertLocked ignores stale timestamps, and a raw SET of a vec: key also refreshes its bq: key.
• build (wasm): the public wormdb module is registered once on narrow-atomics targets. Zig 0.17 panics on a duplicate addModule name, where 0.16 silently replaced it, and that broke zig build wasm-smoke.

Behavior change: replication streams don't reply to one-way writes

• SET, DEL, VINSERT, VDELETE, VBULKINSERT, VRABITQ_INSTALL and EXEC append_log_witness on a replication connection no longer get an OK/ERR frame back.
• The cluster sender never read those replies. They filled the socket buffers until the 2 s send timeout dropped the link.
• Rejected frames are logged instead.
• The request/response RPCs (EXEC proof_prefix_root, EXEC vsearch_local_raw) still reply. The legacy-replication security regression now uses one of them to check that the stream is live and the write landed.

Performance

• server: buffered TCP reads (one 16 KiB read instead of a poll + read pair per frame), SIMD WebSocket unmasking, cache-line-aligned metrics, and a StaticStringMap for EXEC lookup.
• storage: in-place value overwrites, a WAL queue mutex with no allocation under the lock, and a Dekker-style writer wake handshake.
• storage/proof: bounded scan heap, 256-way snapshot merge, and a store-scoped sequence cache.
• vector: vectorized binary quantization, distance kernels and RaBitQ. A lock-free first-byte namespace filter keeps raw SET/DEL from touching the namespace registry for unrelated keys.
• vsearch: HNSW distances are used directly, and raw SET/DEL keep the index in sync.

Behavior change: vsearch ef

vsearch now takes an optional ef argument (<N> or ef=<N>).

• The default beam width is now max(128, 2 × top_k). It used to be 20 × top_k, which is 200 at top_k = 10.
• HNSW results are no longer re-ranked against a larger stage-1 candidate set.
• This is faster but can lower recall on hard data. On a uniform-random 100k set at top_k = 10, recall went from 0.671 to 0.595. Pass a larger ef to trade speed back for recall.

Tooling

• Release workflow: .github/workflows/release.yml adds a multi-platform release. Actions are pinned to commit SHAs. The workflow defaults to contents: read, and only the publishing job can write.
• zig build bench: builds the bench/*.zig micro-benchmarks. It defaults to Debug, so pass -Doptimize=ReleaseFast.
• bench/wire_load_bench.zig: a native WormWire load generator. It runs N connections on one thread each, with M requests in flight per connection. It reports ops/s and p50/p95/p99, and exits non-zero if any operation failed. The Bun bench client tops out below the server under pipelined load, so it can't measure server changes there.

Testing

• zig build test on the tree with the meshguard bump: 331/331 pass on Windows and 332/332 on Linux (Ubuntu).
• The Linux cross-compile (-Dtarget=x86_64-linux-gnu -Dcrypto-backend=std) builds.
• zig fmt --check bench src build.zig is clean.
• The CI engine steps also pass locally on Windows and Linux at the head commit: zig build test, zig build -Doptimize=ReleaseSmall, zig build wasm-smoke -Dtarget=wasm32-wasi, and scripts/test-security.ts (22/22).
• The last commit only adds the load generator, which isn't part of the test suite. It builds and runs on Windows and Linux. It was also checked against a server killed mid-run: it stops, counts the in-flight requests as failed, and exits non-zero.

Benchmarks

All runs are A/B against the commit right before this pass (d054889, the Zig 0.17 migration alone). Builds are ReleaseFast and the server runs on loopback. Base and head alternate order each round, and each head run is compared with the base run next to it. Numbers are median head/base throughput, with rounds won in parentheses.

Linux, pipelined (native load generator, 8 connections × 16 in flight, 100k ops, 8 rounds, 0 failed ops):

Persistence	read	write	exec transfer
none	1.84× (8/8)	2.03× (8/8)	1.84× (8/8)
full	1.92× (8/8)	4.08× (8/8)	1.83× (8/8)

Durable pipelined writes went from about 50k to about 190k ops/s. A separate 10-round run gave the same picture: 1.5–2.1× on read and transfer, and 3.9× on durable writes.

Single client, one request at a time:

• Linux is about 1.0× on every test. A single client is limited by the loopback round trip.
• Windows (Bun client, 5 rounds) is faster with persistence on: head won 4 of 5 rounds on read, write, mixed and status, at a median 1.4–1.9×. With persistence off the gains are smaller, and status is unchanged.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.