Skip to content

Stabilize context allowlist serialization - #1209

Merged
prk-Jr merged 6 commits into
mainfrom
fix/1197-deterministic-context-allowlist
Oct 1, 2026
Merged

prk-Jr merged 6 commits into
mainfrom
fix/1197-deterministic-context-allowlist

Conversation

@prk-Jr

@prk-Jr prk-Jr commented Sep 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Serialize auction context allowlists in sorted order so independently loaded configurations produce the same template fingerprint and config-envelope hash.
  • Preserve the existing array wire format, deduplication, and membership behavior. No configuration migration is required; an existing unsorted envelope may need one final push.
  • Add regressions for independently parsed fingerprints, one origin fetch across repeated template requests, all serialized settings maps/sets, and repeated CLI diff/push processes.

Changes

File Change
crates/trusted-server-core/src/auction_config_types.rs Replace the context allowlist's HashSet with BTreeSet.
crates/trusted-server-core/src/auction/formats.rs Update the allowlist test constructor.
crates/trusted-server-core/src/auction/orchestrator.rs Update the allowlist test constructor.
crates/trusted-server-core/src/settings.rs Update the allowlist parsing test.
crates/trusted-server-core/src/config.rs Check canonical bytes, sorted/deduplicated keys, and envelope hashes across fresh parses with populated nested collections.
crates/trusted-server-core/src/publisher.rs Clarify array-order requirements and test fingerprint stability and actual template-cache reuse.
crates/trusted-server-cli/tests/config_env_overlay.rs Check 16 independent diff/push cycles against an isolated local store.

Closes

Closes #1197

Test plan

  • Regression tests failed before the fix for varying fingerprints, cache keys, serialized settings, and CLI diffs.
  • Native core suite: 2,764 tests passed.
  • CLI config suite: 8 tests passed, including repeated diff --exit-code and no-op pushes.
  • cargo fmt --all -- --check
  • All eight Clippy aliases from AGENTS.md (Fastly, Axum, Cloudflare native/WASM, Spin native/WASM, CLI, codegen).
  • cargo test-fastly, cargo test-axum, cargo test-cloudflare, and cargo test-spin.
  • Cross-adapter parity suite: 14 tests passed.
  • JS build and Vitest: 47 files / 1,130 tests passed using Node 24.12.0.
  • JS and docs formatting checks.

Fastly tests used SSL_CERT_FILE=/etc/ssl/cert.pem because the sandbox has no macOS keychain. Axum tests passed with loopback socket access after the sandbox denied local test-server binds.

A live-origin smoke check confirmed page and injected script delivery, but that page's cookie-setting responses bypass shared caching. Cache reuse for this fix is verified by the automated publisher regression: 16 independently parsed configurations reuse one template and fetch the origin once.

Checklist

  • Changes follow AGENTS.md conventions.
  • No new production unwrap(), logging, or dependencies.
  • New behavior has regression coverage; independent review found no actionable issues.
  • No credentials, operator configuration, or private runtime data included.

@prk-Jr prk-Jr self-assigned this Sep 24, 2026
@prk-Jr prk-Jr added this to the 202609 milestone Sep 24, 2026
@prk-Jr
prk-Jr requested a review from dhruv8sh September 24, 2026 10:16
Comment thread crates/trusted-server-cli/tests/config_env_overlay.rs
Comment thread crates/trusted-server-core/src/publisher.rs
Comment thread crates/trusted-server-core/src/auction_config_types.rs
Comment thread crates/trusted-server-core/src/config.rs

@dhruv8sh dhruv8sh left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Switching allowed_context_keys to a BTreeSet fixes the root cause of #1197: the only unordered array in the serialized settings. EdgeZero's envelope hash already sorts object keys, so this makes both the template fingerprint and ts config push/diff stable. The array wire format and membership semantics are unchanged, and the regression tests reproduce the bug at three layers. Nothing here blocks merge.

2 of the inline comments below carry a one-click GitHub suggestion — use Commit suggestion (or Add suggestion to batch for several at once) to apply them as commits on the PR branch. The CHANGELOG note below is prose because CHANGELOG.md isn't part of the diff.

Non-blocking

🤔 thinking / ⛏ nitpick

  • Fingerprint still depends on serde_json/preserve_order being off — see inline at crates/trusted-server-core/src/publisher.rs:2086
  • Document the sorted, deduplicated serialization — see inline at crates/trusted-server-core/src/auction_config_types.rs:77

👍 praise

  • Regression coverage at three layers — see inline at crates/trusted-server-core/src/publisher.rs:11754

Cross-cutting / body-level findings

  • 🏕 CHANGELOG entry — This is visible to operators: existing envelopes with two or more allowed_context_keys show one ts config diff change after upgrade and need one more push, and Fastly deployments using ESI assembly get their template-cache hits back. Something like this under ## [Unreleased] → ### Fixed in CHANGELOG.md (apply manually — CHANGELOG.md isn't in the diff):

    - `[auction].allowed_context_keys` now serializes in sorted, deduplicated order, so template-cache fingerprints and `ts config push`/`diff` envelope hashes stay the same when one configuration is parsed more than once. Previously, configurations with two or more keys could get a new template-cache key on every Fastly request under ESI assembly, and `ts config diff` could report a change for an unchanged file. Existing envelopes may show one final diff; push once after upgrading.

CI Status

  • cargo fmt: PASS (required)
  • cargo test: PASS (required)
  • format-typescript: PASS (required)
  • format-docs: PASS (required)
  • cargo test (axum native): PASS
  • cargo check (cloudflare native + wasm32-unknown-unknown): PASS
  • cargo check/build/test (spin native + wasm32-wasip1): PASS
  • cargo test (ts CLI, native): PASS
  • cargo test (cross-adapter parity): PASS
  • vitest: PASS
  • integration tests: PASS
  • integration tests (Fastly EC lifecycle): PASS
  • browser integration tests: PASS
  • prepare integration artifacts: PASS
  • CodeQL: PASS
  • Analyze (rust): PASS
  • Analyze (javascript-typescript): PASS
  • Analyze (actions): PASS
  • CLAUDE.md symlink guard: PASS

Comment thread crates/trusted-server-core/src/publisher.rs Outdated
Comment thread crates/trusted-server-core/src/auction_config_types.rs Outdated
Comment thread crates/trusted-server-core/src/publisher.rs

@ChristianPavilonis ChristianPavilonis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Reviewed 29b1cfe31ff7d64756d5aa2864d7bdbd3d3649c1 against 56a19229f1e1e0369efbfe86d3a58b87e9394f7b. The BTreeSet change makes allowed_context_keys serialization deterministic while preserving deduplication, membership filtering, and the array wire format. Focused tests passed for serialization, auction filtering, template-cache reuse, repeated CLI diff/push behavior, and formatting. No actionable change-introduced issues found.

prk-Jr and others added 4 commits September 30, 2026 18:45
Use EdgeZero canonical hashing for template fingerprints so JSON map ordering does not depend on serde_json features. Correct the envelope regression assertion, improve CLI diagnostics, and document serialization and upgrade effects.
@prk-Jr
prk-Jr merged commit cb94525 into main Oct 1, 2026
20 checks passed
@aram356 aram356 modified the milestones: 202609, 202610 Oct 12, 2026
@aram356
aram356 deleted the fix/1197-deterministic-context-allowlist branch October 15, 2026 17:06
@aram356 aram356 mentioned this pull request Oct 23, 2026
15 of 16 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Template cache keys and config pushes vary per load when allowed_context_keys has 2+ keys

4 participants