Repository navigation
turnloop-http: depend on brotli-decompressor instead of brotli (decoder only) #112
Description
turnloop-http depends on the umbrella brotli crate, but only ever decodes. Switching it to the decode-only brotli-decompressor would drop the Brotli encoder from the dependency graph of every consumer.
Evidence (turnloop-http 0.1.0-alpha.8, brotli 8.0.4, brotli-decompressor 5.0.3)
turnloop-http calls only decoder APIs. Every brotli:: reference in src/:
brotli::Allocator
brotli::BrotliDecompressStream
brotli::BrotliResult::{NeedsMoreInput, ResultFailure, ResultSuccess}
brotli::BrotliState
brotli::HuffmanCode
brotli::SliceWrapper
brotli::SliceWrapperMut
No encoder type or function appears anywhere.
The umbrella crate gets its decoder from brotli-decompressor anyway. brotli/src/lib.rs has extern crate brotli_decompressor; and re-exports its decode surface. brotli-decompressor exports every symbol listed above directly, so this should be a manifest change plus use-path edits, with no logic change.
The encoder is most of the crate. brotli-8.0.4/src/enc/ is 4.2 MB of the crate's 4.4 MB of source; brotli-decompressor is 2.0 MB.
Why it matters downstream
Perry's fetch decodes Content-Encoding: br through turnloop-http, so every Perry program that uses fetch compiles the whole Brotli encoder to obtain a decoder. A Perry binary-size audit attributes ~800 KB to the encoder. How much of that survives a real link after dead-stripping is being measured separately, but removing it at the dependency level makes the result independent of any particular linker's stripping, which is the more robust property.
Suggested change
[dependencies]
brotli-decompressor = "5"in place of brotli = "8", with the use brotli::… paths updated.
If turnloop-http ever needs to encode (for example a server-side response-compression path), an opt-in brotli-encode feature pulling the full crate would preserve that without charging decode-only users for it. There is no encoder use today.
Activity
proggeramlug commentedon Sep 23, 2026
Measured follow-up — the change here is still correct, but on its own it will not remove the encoder from Perry's fetch binaries, and I want to correct the impact I implied.
Three-arm measurement on Perry main
1dbe9f46ed(macOS arm64, auto-optimize path), bytes attributed via the linker map (PERRY_LINK_MAP), so these are symbols that survived-dead_strip:async-runtimeasync-runtime, web-fetchzlib.brotliCompressSync+ external-zlib-pumpThe encoder is live in a fetch-only program (868 symbols, including real functions such as
BrotliFindAllStaticDictionaryMatchesandCreateCommands, not just tables). ForcingbrotliCompressSyncadded nothing, because it was already all present.turnloop-http is not what pins it. turnloop-http calls no encoder code, so anything reachable only through it is strippable. The pin is on Perry's side: Perry's
web-fetchfeature implies its bundled WHATWG streams module, and that module'sCompressionStreamhandles every format in one runtimematchthat includesbrotli::CompressorReader::new— 26CompressorReadersymbols are live. A runtime match arm cannot be dead-stripped while its function is reachable.So: this issue remains worthwhile hygiene — turnloop-http has no reason to depend on the encoder, and it stops other embedders compiling it — but the 823 KB in Perry needs a Perry-side fix to its streams feature split. No change to the suggestion above; just the expectation of what it buys.
proggeramlug commentedon Sep 23, 2026
Correction to my previous comment — its numbers were contaminated. Both fetch arms there had silently fallen back to Perry's prebuilt full-feature runtime: a Perry-side compile break (
base64missing from a feature, now fixed in PerryTS/perry#11116) made auto-optimize fall back and exit 0. So "fetch-only 23.63 MB" was really the full archive.Re-measured with the fallback fixed and an assertion that the stripped runtime was actually built:
zlib.brotliCompressSyncThe conclusion in my previous comment still holds, now on clean evidence: the encoder is live in a genuinely stripped fetch-only program, and it is pinned by Perry's own streams module (
web-fetchimplies the bundled WHATWG streams, whoseCompressionStreamdispatch includesbrotli::CompressorReader), not by turnloop-http. So this issue is still worthwhile hygiene but won't by itself remove it from Perry.The third row shows a separate Perry-side duplication — two copies of the encoder when a program uses both — which is also not a turnloop concern.