Description

TS cacheable-root architecture: ESI and the alternatives

Date: 2026-08-06
Context: Trusted Server on www.prospect-a.com (prospect-a), live behind a ts-tester=true cookie + basic-auth gate. TS v394, Next.js 16.2.9 App Router, Fastly Compute.
Status: Design exploration (brainstorm). Not a committed plan.
Redaction: Publisher redacted to prospect-a / prospect-a.com. Safe for internal sharing; still confirm before any public posting (GitHub, Slack outside the deal channel, Artifacts).

Numbers labeled measured come from the live A/B below (N=4 per side, unthrottled, single US edge, one session). Numbers labeled modeled are estimates, not observations.

Part 1. Findings: TS-on vs TS-off (shareable)

We ran a tester-cookie A/B on the same URLs in isolated browser contexts: cookie on = TS/SSAT path, cookie off = plain cached origin. Same URL and auth both sides. Measured on the homepage and on a news article that was new to the tester.

Timing (measured, ms)

Cache behavior (measured, from document response headers)

Same article, two paths:

• TS-off: hit-state: MISS-CLUSTER but x-cache: HIT, MISS with age: 14110 → the edge PoP missed, the Fastly shield had it cached (~4h old, from other traffic). Server time 165 ms. Cacheable. First load in a fresh context was 650 ms (shield fetch), then the PoP cached it and subsequent loads dropped to 65–230 ms.
• TS-on: hit-state: PASS, x-cache: MISS, MISS, cache-control: private, max-age=0, x-ts-version: 394. Server time 773 ms. Never shared-cached; pays the server-side auction on every load.

What it means

TS's measurable cost is TTFB, roughly +600 to +800 ms, and that is the server-side auction (SSAT) running inline before the first byte. Time-to-fully-painted ads is comparable on and off, because TS shifts the auction cost from client-side (post-load) to server-side (pre-first-byte) rather than adding new work. Fill was reliable in all 16 measured loads.

Two secondary findings:

• A URL new to the tester does not defeat the cache. The Fastly cache is shared; a live article is already warm at the shield from other traffic. Only a cache-buster query would force a cold origin build.
• TS's googletag shim discards event listeners queued before it loads, so third-party tooling that hooks googletag slot events before TS (viewability, some analytics) may not receive them on the TS build. Flagged for follow-up; not yet filed.

The optimization target this doc addresses: the root HTML is uncacheable under TS (PASS) while the same page off-TS is a shared HIT. If we can make the root cacheable again and keep ad slots per-user, we recover most of the ~600–800 ms.

Part 2. The goal

Cache the static root as a shared Fastly HIT (like the origin page), and deliver the per-user parts (bids, identity) out of band. That recovers the TTFB while preserving the first-party server-side auction.

It is reachable, and not by accident. Four enablers already exist in the codebase:

• The auction is already decoupled from first byte: it is dispatched early (async) and only the </body> tail is held for bids (publisher.rs:2723, :2214).
• The per-user payload is already isolated to two injection seams, tsjs.adSlots at <head> open and tsjs.bids/adInit() before </body> (html_processor.rs:300, :344), not smeared through the DOM.
• A fragment endpoint already exists: /_ts/page-bids returns bids as JSON out-of-band (publisher.rs:3550).
• A placeholder/token-substitution engine already exists: the Next.js RSC placeholder rewriter swaps payloads for __ts_rsc_payload_N__ tokens during streaming and restores them in post-processing (rsc_placeholders.rs, html_post_process.rs).

Two blockers stand in the way:

1. The </body> hold welds the bid tail onto the same HTTP response, so the root and the per-user bids are one cache entry. The root can't cache until bids leave the response body.
2. The cookie-privacy net force-privatizes any Set-Cookie response (response_privacy.rs:31, re-applied at send in the Fastly adapter main.rs:338), and the EC identity cookie rides the first navigation. A cacheable root must be served without setting that cookie inline; identity has to move to a subrequest or edge-KV path.

Both blockers are prerequisites for any of the approaches below, ESI included.

Part 3. Approaches (three ways to the same goal)

All three serve the Part 2 goal. They sit at different layers and are not mutually exclusive.

Lead: ESI via Fastly's esi crate

Templatize the root once (on cache MISS): TS's lol_html pass emits <esi:include src="/_ts/page-bids?..."> (and per-slot includes) at the two seams instead of inlining per-user data. The resulting ESI template carries no per-user data and is shared-cacheable. On every request, Fastly's esi crate processes the cached template, fetches the per-user fragments (the bids fragment is the SSAT auction), and streams the assembled page. The fragment fetch overlaps streaming the cached root.

Alternative A: cache-root + client-fill (no ESI)

Same cacheable templatized root, but the holes are filled by a small inline script that fetches /_ts/page-bids (which already exists) and resolves identity, client-side, after the root loads. No edge assembly, no new dependency, portable across all adapters.

Alternative B: shimmed React (correctness layer, not a transport)

Intercept React hydration to reconcile TS-rewritten URLs and to render ad slots from a late-arriving bids payload without a full re-render. This is the correctness layer that makes a static/dynamic split hydration-safe on the App Router, and it fixes the existing hydration mismatch (issues doc #1). It complements, it does not replace, whichever transport we pick.

Comparison

Recommendation

• Spine on Fastly is ESI, scoped to the bids/ad holes only (see Part 5, option 2). It has the highest ceiling on the perf win because assembly stays at the edge with no client round-trip.
• The portable fallback is client-fill via /_ts/page-bids for the non-Fastly adapters, and the baseline if the portability decision goes against ESI.
• Layer the React shim as the correctness fix on top of whichever transport, since the hydration mismatch has to be solved regardless.
• Defer full RSC/flight partitioning. The T-chunk length-preservation logic (rsc.rs) has no static/dynamic split today; carving cacheable vs per-user flight chunks is a separate, larger effort.

Part 4. ESI deep dive (developer view)

The crate (esi 0.7.1, June 2026, MIT)

A streaming ESI parser and executor built for Fastly Compute, implementing a subset of Akamai ESI 5.0.

• Two modes: Processor::process_response() (buffered; allows response-manipulation functions like $add_header and auto-emits Cache-Control) and Processor::process_response_streaming() (streams body, commits headers early, low TTFB, drops the response-manipulation functions).
• Config via Configuration: with_chunk_size(usize), with_caching(CacheConfig) (caches rendered output and fetched fragments), with_function_recursion_depth(usize).
• Fragment sub-requests go through a caller-provided closure (req.with_ttl(120).send_async("backend")), so we control routing and can issue them concurrently. An optional second callback processes fragment responses before streaming.
• Tag support relevant to us: esi:include, esi:vars, esi:assign, esi:choose/when/otherwise, esi:try/attempt/except, alt and onerror="continue", $(VAR) interpolation, request-control attributes (ttl, no-store, maxwait, method).
• Requirements: the fastly crate (~0.12), so it is Fastly-Compute-only. Viceroy for tests.

The two-stage mechanism

Today TS does one streaming pass: fetch origin → lol_html rewrite → inline per-user data → hold </body> for bids → stream. That is one HTTP response and one (private) cache entry.

Under ESI it becomes two stages with a cache boundary between them:

1. Templatize (cache MISS only). Fetch origin, lol_html-rewrite to emit <esi:include> at the two seams instead of inlining. Cache the resulting ESI template (shared, no per-user data).
2. Assemble (every request). Run the esi crate over the cached template; it fetches the per-user fragments (bids = the auction) and streams. Because the include sits low in the document, the fragment fetch overlaps streaming the cached root.

Steady-state per-request cost drops to ESI assembly + one small auction fragment. The origin fetch and full-document rewrite happen only on MISS. That is where the modeled TTFB recovery comes from.

What it touches

• Generalize the placeholder rewriter (rsc_placeholders.rs, html_post_process.rs) to emit esi:include markers at the two seams (html_processor.rs:300, :344).
• Make the templatized root cacheable: relax the should_run_ad_stack privatization (publisher.rs:2882) for the templated path, and stop setting the EC cookie inline.
• Add the esi crate as a second-stage processor on the Fastly adapter, with the current inline path retained as the non-Fastly fallback.
• Move identity resolution to a fragment or edge-KV path (ec/ module).

Hard parts

• Identity off-inline is a hard prerequisite. The privacy net will force-privatize the root otherwise, gate be damned (main.rs:338).
• Two streaming rewriters in series (lol_html then esi), both !Send and chunk-oriented. Sequencing plus the cache boundary between them is the real integration cost.
• Cache-key / vary discipline. The origin varies on rsc, next-router-*, prospect-a-Exp (A/B experiments). The template cache key must honor those or serve the wrong variant.
• Streaming mode drops ESI response-manipulation, so TS keeps setting its own headers around the processor. Minor.

Feasibility verdict: feasible and architecturally aligned, not a moonshot. The seams already exist.

Part 5. ESI options and the tradeoff (VP of product view)

Three injection models

1. Publisher-authored ESI (prospect-a emits the tags). Rejected: pushes work onto the publisher and breaks the "drop TS in front, no origin changes" promise.
2. TS-injected ESI, bids/ad holes only (TS templatizes the two seams, caches the root, ESI-fills bids). This is the recommended scope: the smallest change that captures the TTFB win.
3. TS-injected ESI, full dynamic partitioning (also carve RSC/flight, identity, consent into fragments). Defer; much larger, new work on rsc.rs.

Positive view

This kills the single biggest objection we measured against TS: the ~600–800 ms TTFB tax from running the auction inline. It converts "TS makes the page slower" into "the TS root caches like origin, ads stay per-user," which is exactly the publisher's mental model: cache the article, personalize the ads. It does so with Fastly's own, now-improved, supported crate rather than bespoke assembly code, which is on-platform and easier to defend in review. And it strengthens the core story: keep the first-party server-side auction and identity, lose the perf cost. For a performance-sensitive publisher like prospect-a that is a materially better pitch.

Negative view

• Portability. The esi crate is Fastly-Compute-only. TS is deliberately multi-adapter (Fastly, Cloudflare, Spin/Akamai, Axum), and the EdgeZero portability story is a stated strategic asset. An ESI assembly path is Fastly-exclusive, so we would maintain two rendering architectures and the flagship perf number would exist only on Fastly. That cuts against "run anywhere."
• Dependency risk. This puts the critical render path on a 0.7.x crate (young, ~80% documented). If Fastly deprioritizes it, we own a fork.
• Privacy surface. Caching a root that must be per-user-identical is a deliberate exception to the cookie-privacy net that exists to prevent per-user leakage into shared cache. It has to be airtight and continuously tested.
• Priority. This is a perf optimization, not a correctness fix. The inline path already fills reliably (6/8, 7/10 measured). It competes for eng time against open correctness issues (hydration Initialize repo #1, config Init Github actions #3, the SSAT 100x price bug).
• Operational weight. Cache invalidation on article edits, consent changes, and experiment variants is now our problem.

The gating decision

Is Fastly-first acceptable for the flagship perf path, with the inline/client-fill path as the portable fallback?

• If yes: scope 2 is a strong, aligned bet; green-light a spike.
• If portability parity is non-negotiable: ESI stays a Fastly-only accelerator layered on a portable inline baseline. Fine, but it can't be the architecture, only a per-platform boost, and the client-fill approach (Alternative A) becomes the primary way to recover TTFB everywhere.

Part 6. Open questions and next steps

1. Portability decision (blocking). Fastly-first-with-fallback, or portability parity? Everything downstream depends on this.
2. Identity re-architecture (prerequisite for all approaches). Move EC cookie issuance off the inline navigation response to a fragment/edge-KV path. Needs its own design.
3. Cache-key/vary strategy. Enumerate what the templated root must vary on (rsc, next-router-*, prospect-a-Exp, geo, consent state) and confirm none of it is per-user.
4. Spike (modeled).

5. Validation. The ~600–800 ms recovery is modeled. Confirm it with the same tester-cookie A/B harness once a spike serves a cached templated root: expect TS-on TTFB to approach the TS-off shield-HIT numbers (~65–230 ms warm).

Future-state options (not recommended now)

• Full RSC/flight partitioning (option 3) once the bids-fragment ESI path is proven.
• Fastly Varnish shielding / classic ESI if the topology ever puts Varnish in front of Compute, which would change the portability calculus.

Appendix: code-grounded seams (file:line)