Skip to content

Add page changes as middleware - #1295

Closed
Jamesr51d wants to merge 432 commits into
IABTechLab:mainfrom
jwrosewell:split/10-page-middleware
Closed

Jamesr51d wants to merge 432 commits into
IABTechLab:mainfrom
jwrosewell:split/10-page-middleware

Conversation

@Jamesr51d

@Jamesr51d Jamesr51d commented Oct 12, 2026 •

Copy link
Copy Markdown

Stacks on the inspection endpoints (#1294). The additions that follow the seven pull requests open here are listed in #777.
Compare split/9-visibility-endpoints with split/10-page-middleware
to see only this pull request's change.

It builds section 3.7 of the seam design (#1084).

A module changes a page through a middleware and in no other way, and a middleware runs only on the pages where an ordered entry of the settings names it. Every module that changed a page is converted, core's own script proxy included, and the four page traits are removed.

What this pull request does

[[fetch]]
media_type = "text/html"
path = "/news/"
middleware = ["tag.example.cleanup", "tag.example"]

[[fetch]]
media_type = "text/html"
middleware = ["tag.example"]

[[serve]]
media_type = "text/html"
middleware = ["tag.example.reader"]

Each phase has an ordered list of entries. An entry covers text/html, optionally only the requests under a path prefix, and names the middleware to run in order. A page takes the first entry that covers it. An entry places and orders, and switches nothing on. The section that selects a module still decides whether it runs.

Phase Runs Told about the reader Stored in a shared template
[[fetch]] On the page as the origin sent it No Yes
[[serve]] On each reader's copy, whether the page came from the store or from the origin Yes No

The contract is trusted_server_core::middleware. A middleware is asked once for each document and answers with any of five things.

Part What it does
Head markup Written in <head> ahead of the script bundle
Markup for after the bundle Written straight after the bundle's tag
Element handlers For the elements a selector matches, keep the element, replace one attribute's value or remove the element
Text handlers For the text inside the elements a selector matches, keep a chunk, replace it or remove it
Stream processor Runs over the document the handlers left

A registration offers one with .with_middleware(...). The guide has two fixtures a vendor can copy, and the documentation test compiles both as written.

What converted

Each conversion is one commit. The first commit of this pull request recorded the page each module produced through its hooks, as a file in the module's crate, and every conversion leaves its recording byte for byte as it was.

Module Through hooks it had Now supplies
Prebid An attribute rewriter and a head injector auction.prebid, fetch
JavaScript asset proxy, in core An attribute rewriter js_asset_proxy, fetch
Testlight An attribute rewriter testing.testlight, fetch
Next.js Two script rewriters and a stream processor framework.nextjs, fetch
Permutive An attribute rewriter audience.permutive, fetch
lockr An attribute rewriter identity.lockr, fetch
Didomi A head injector cmp.didomi, fetch
Sourcepoint An attribute rewriter and a head injector cmp.sourcepoint, fetch
Google Tag Manager An attribute rewriter and a script rewriter tag.google-tag-manager, fetch
DataDome An attribute rewriter, and a head injector that reads what its request filter decided bot-protection.datadome, fetch, and bot-protection.datadome.tag, serve
Google Publisher Tags An attribute rewriter and a head injector ad-tag.google, fetch
GPT diagnostics A head injector that reads what its request preparer decided ad-tag.google.diagnostics, serve
APS A head injector that wrote no markup and set one attribute on the bundle's tag No middleware. It states the attribute with with_bundle_tag_attribute

Twenty implementations of the four traits became thirteen middleware, because a module's hooks that work on one page are one page change.

Comments in prebid/index.ts and the Osano test that named a head injector name the module's middleware.

Where the serve entry runs

A page leaves from four places, and the serve entry runs at each.

  1. A page that is not stored runs it on the same pass as the fetch entry, after it, so a streamed page is still streamed.
  2. A page transformed as a shared template runs it once the template is stored and this reader's copy is assembled.
  3. A page from the store runs it after assembly in the buffered finalizer.
  4. A page from the store runs it piece by piece in the streaming finalizer, so the head still goes out before the auction is awaited.

A page from the store keeps no record of its origin's host. A serve middleware is told the host from the settings, so it is told the same on a hit as on a miss.

What is refused, and when

When What is refused
When the settings load, and at deploy validation An entry for a media type other than text/html, a path that is not a request path prefix, an entry that names nothing or names one middleware twice, a key an entry does not read, an entry an earlier one already covers, and two entries for the same pages, which are told to share one list
When the service starts An entry naming a middleware that no running module supplies, with the names that could be written. An entry naming a middleware in the phase it does not run in. Two registrations that supply one name
When a page is served A handler whose selector does not parse. The response fails with a 500 that names the middleware, before a byte of the page is sent, where the parser would otherwise panic

A name is checked when the service starts and not at deploy validation, which is where a section's module names are checked and for the same reason. A crate the ts tool never links may supply the name.

Three faults found on the way

Recording the pages showed two faults in how a rewritten inline script is written back, both on main as well, and CI on Windows showed a third. Each has tests that fail without its fix.

  1. A script a rewriter replaced was written back as text, which escapes &, < and >. A browser decodes no entity inside a <script>, so the standard Google Tag Manager snippet's '&l='+l reached the page as '&amp;l='+l, and a loader holding && or < became a syntax error. It reached a rewritten __NEXT_DATA__ holding those characters too.
  2. The Google Tag Manager module holds a script's text back while a marker might still arrive. A part that arrived after the held text had stopped looking like a marker went out ahead of it, so a script such as footer.innerHTML = '<b>Search with google</b> <i>or</i> <u>another</u> engine'; reached the page out of order.
  3. The GPT bootstrap script is embedded with include_str! and written into the page. A checkout that rewrites line endings changed the bytes of the page, which a Windows runner caught. The file is pinned to one set of line endings.

The first two commits stand alone and can be taken on their own. Core moves the origin's address in srcset and imagesrcset, and no test said so. A test now does, added before the hooks were removed and passing on both sides of the removal.

Why

A module changed a page through four hook traits, IntegrationAttributeRewriter, IntegrationScriptRewriter, IntegrationHtmlStreamProcessorFactory and IntegrationHeadInjector. Every hook of a selected module ran on every page, in the one order the modules were registered in. A publisher could not run a change on part of a site, could not choose the order, and could not place two changes of one module apart. Whether a hook's output could be stored in a shared template rested on what the hook declared about itself.

What breaks

A deployment that selects a module and writes no entry gets no page change from it, and startup logs a warning that names each middleware no entry names.

  • trusted-server.example.toml documents one [[fetch]] and one [[serve]] entry naming every middleware the stock modules supply, in the order the modules' hooks ran in. A deployment keeps the names of the modules it selects.
  • ts audit generate writes the entries for the modules it selects, in that order.
  • crates/trusted-server-modules lists the stock middleware (middleware(), middleware_for(...)). Its tests hold the list to what the modules register, and hold the example configuration and the configuration guide's table to the list.

One order is worth knowing. The script proxy knows a script by the address it is handed. It ran ahead of every module a section selects, and it now runs where the entry puts it, so it is named ahead of a middleware that moves the address of a script it is to proxy or block.

Removed:

  • IntegrationAttributeRewriter, IntegrationScriptRewriter, IntegrationHtmlStreamProcessorFactory and IntegrationHeadInjector, with their four context types.
  • The registration methods that took them, and the registry's four lists.
  • AttributeRewriteOutcome and ScriptTextAccumulator. A text handler is made for one document and keeps its own text, so the per-document buffer that guarded against one document's text reaching the next has nothing left to guard.
  • The function that handed a shared template no request state. What a request left reaches a document through the [[serve]] entry alone, which does not run on the pass that builds a template every reader shares, and a fetch middleware is handed none of it. This is the rule section 3.7 said would become structural.

IntegrationMetadata carries a middleware list, the names of the middleware the integration supplies, in place of attribute_rewriters, script_selectors and head_injectors.

The document processor no longer asks the registry anything about a page. It moves the origin's address in the five attributes it always has, writes the bundle, and carries out the plan of the middleware an entry names.

How it was verified

CI passed on the fork on the head, e8239dac3. CI has not run here, because the branch conflicts with main.

Closes #1269.

Every name an operator types into configuration is snake_case, so the
demonstration provider is now selected with `[ec] provider = "client_fixed"`.
The key constant, the two startup messages that spell the key, the page
script's log lines, the cargo feature comment, the provider table in AGENTS.md
and the resolve endpoint in the API reference all follow. This commit changes
only that one name.

The cargo feature that compiles the provider in keeps the name
`client-fixed-demo`, because a cargo feature name is not a configuration value.
The tsjs module id `ec_client_fixed` is unchanged as well, being already
snake_case and never compared against the key.

The old name is refused when settings load, which every adapter does through
`settings_from_config_blob` before it serves a request. The message names
`client_fixed` and says what to write, so a deployment still configured with
`client-fixed` stops at startup rather than being told to add a provider block
that this provider does not need. A block written under the old name does not
make the old name acceptable either, because the name is refused before any
block is looked for. That is a breaking change, accepted for a major release.

Tests. `cargo fmt --all -- --check` is clean and `cargo clippy -p
trusted-server-core --all-targets --all-features -- -D warnings` passes. The
core suite passes 2,886 tests and 8 doc-tests, with and without the
`client-fixed-demo` feature, the new test among them. The Axum adapter passes
62 tests, `cargo clippy-axum` is clean, and `cargo check -p
trusted-server-adapter-fastly --target wasm32-wasip1` builds. The page
script's own vitest file passes 6 tests and ESLint is clean on it.
Every provider type selects the same way, with provider in its own section,
and names its providers in snake_case. Permission signals now follow that,
so [permission_signal] sources becomes [permission_signal] provider. The key
stays an ordered list, and a configuration that leaves it out still runs
every provider the adapter links, in the order the adapter offers them.

The two hyphenated provider names become gpp_sale_opt_out and us_privacy,
while gpc and tcf are unchanged. The crates report the new identifiers, and
core's selection messages, the startup log, the module README, the guides,
the example configuration and the permissions sample follow.

sources is removed rather than accepted alongside provider, which breaks a
configuration written against the branch that introduced it. A section
carrying sources is refused when the settings are read, with a message
naming provider, on each path a deployment reads settings through, being a
TOML file, the TOML value ts config push reads, and the JSON config blob
read at startup. Reading the section by hand rather than with
deny_unknown_fields is what makes that message possible, because a derived
refusal can only name sources by declaring it as a field, and would then
offer it as a key it expects whenever it refused any other.

No provider takes settings, so a [permission_signal.<name>] block stays an
unknown field. Its refusal now says that a provider which gains settings
will take them there.

Tests cover the removed key on each of those paths, a settings block
refused as an unknown field, a list surviving a config blob round trip,
and, where the real crates are linked, the documented names selecting all
four providers in order and an old hyphenated name being refused with the
names available.
An integration used to carry its own `enabled` flag, with a default that
differed between integrations, so whether one ran depended on the flag, on
that default and on whether its block was present at all. It now runs when,
and only when, `[integration] provider` names its id, which is the convention
the other provider types already follow.

The `[integrations]` table becomes `[integration]`, and a configuration still
carrying the old name is refused with the move spelled out rather than with a
bare unknown-field error. Every integration's `enabled` field, its default
function, `IntegrationConfig::is_enabled` and the disabled-schema machinery go
with it. A block is written only for an integration the list names: one
written for an integration it does not name is refused, as is an id named
twice and an `enabled` key left behind in a block. A named integration with no
block of its own is read from an empty one, so an integration that takes no
settings runs on its id alone and one that requires a setting reports the
setting it is missing.

The registry builds only the builders the list names, so an integration runs
exactly when an operator names it, whatever its builder would otherwise make
of the settings, and it refuses an id no builder in the deployment supplies,
listing the ids it has. Deploy validation deliberately does not make that
check, because a vendor crate the CLI never links may supply the id. The
DataDome secret paths move with the table, from `integrations.datadome.*` to
`integration.datadome.*`, keeping their optional segments so an integration
that does not run is never asked for a secret.
`ts audit` used to flip an `enabled` key in the four active integration stubs
the starter template carried. It now writes `[integration] provider` with the
integrations it found and can configure on its own, appends the Google Tag
Manager block with the container it read from the page, and appends the
JavaScript asset proxy block with every asset it found set to
`proxy = "disabled"` for review. It names that asset proxy only when it found
at least one asset, because a block with no assets is refused.

`ts prebid bundle` reads and writes `[integration.prebid]`, the
integration-test fixture names the one integration that environment runs and
drops the blocks for the ones it does not, and the template-cache harness
names Prebid in the list rather than flipping a key. The environment override
examples move to `TRUSTED_SERVER__INTEGRATION__<ID>__<SETTING>` and lose the
`ENABLED` ones, because the provider list is an array and the overlay replaces
only scalar leaves that already exist. The browser Prebid module's operator
message points at `[integration.prebid.bundle]`.
Every name an operator types into configuration is snake_case, so the built-in
host-signal provider is now selected with `[ec] provider = "host_signals"` and
configured with `[ec.providers.host_signals]`. The key constant, the serde name
of the typed block, the validation error key, the registered secret path
`ec.providers.host_signals.passphrase`, the placeholder-secret report, the two
startup messages that spell the key and the provider's own log line all follow.
This commit changes only that one name.

The `HostSignals` trait, the `host_signals` fields and arguments that carry it,
and the prose that calls this the host-signal provider are unchanged, because a
host capability is not a configuration value. The test secret-store key name
`host-signals-passphrase-key` is unchanged for the same reason, being a name an
operator picks for a secret rather than a provider name.

The Rust field name and the configuration name are now the same word, so the
`#[serde(rename)]` on the typed block goes and the field is declared the same
way as `hmac` beside it. That also retires the reason the `EcProviders`
validation keys were spelled out by hand, so the note above those keys is
rewritten to say what still holds, which is that each key has to match the
secret path `TrustedServerAppConfig::secret_fields` registers.

The old name is refused when settings load, which every adapter does before it
serves a request. The message names `host_signals` and says what to write. The
refusal comes before any block is looked for, because a block left behind under
the old name is captured as a vendor block, so the old selector would otherwise
find that block, pass the settings check, and fail later in provider resolution
with a message about an adapter that supplies no such provider. That is a
breaking change, accepted for a major release.

Tests. `cargo fmt --all -- --check` is clean and `cargo clippy -p
trusted-server-core --all-targets --all-features -- -D warnings` passes. The
core suite passes 2,764 tests and 5 doc-tests with 4 ignored, the new test among
them. The Axum adapter passes 43 tests across its three binaries, `cargo
clippy-axum` is clean, and `cargo check -p trusted-server-adapter-fastly
--target wasm32-wasip1` builds.
The Edge Cookie provider blocks move from [ec.providers.<name>] to
[ec.<name>], so identity follows the one convention every provider type
uses, where [<type>] provider = "<name>" selects and [<type>.<name>]
holds that provider's settings. The [ec.providers] table is gone, and a
configuration still carrying it is rejected with the new location in the
message.

A block exists only when the provider has settings. The built-in hmac
provider has a required passphrase, so selecting it still needs
[ec.hmac], while a provider with no settings needs no block at all. Only
the adapter that injects a provider knows whether that provider has
settings, so core no longer demands a block for a name it does not
supply itself.

A block may name the implementation it configures with
implementation = "<id>", which makes the block's own name a label of the
operator's choosing. [ec] provider = "primary" with [ec.primary] holding
implementation = "hmac" and a passphrase configures the built-in
provider under a name that means something to the deployment. Everything
that resolves the selection now reads the implementation rather than the
label, covering the built-in lookup, the matching of a provider the
adapter injects, the check that a selected implementation has the
settings it needs, and the errors. An implementation this deployment
cannot build fails startup naming the implementations it does have.

The fixed [ec] keys stay reserved and cannot name a provider, every
other key in the section has to be a table, and a key that is not one is
reported as the unknown field it almost certainly is, so a typo such as
ec_stor is still caught with a sensible message. Provider names and
implementation ids are snake_case. A block the selector does not name
still fails startup, as it did before.

Secrets follow the blocks. TrustedServerAppConfig::secret_fields now
lists ec.hmac.passphrase, and EdgeZero's path segments cannot say
"whatever name the operator chose", so core reads the labeled blocks out
of the configuration itself through the new ConfiguredSecretFields trait
and resolves their passphrases from trusted_server_secrets in the same
pass. Push-time validation, where those fields hold key names rather
than secrets, no longer runs the passphrase value check against a
labeled block's key name. The check itself is unchanged wherever
settings are loaded with their secrets resolved.

The legacy [ec] passphrase shim still works and now points at [ec.hmac].
Core named three vendors. The auction plan compiled a fixed list of
profiles, `standard`, `prebid-server` and `aps`, and would accept only
one ad server, the demonstration mock. A vendor could not ship a demand
source without editing core, which is the coupling the integration seam
exists to remove.

Demand and the ad server are now provider types, configured the way the
Edge Cookie, location, device and permission signal providers are:

    [demand]
    provider = ["pbs_main"]

    [demand.pbs_main]
    implementation = "prebid_server"
    endpoint = "https://pbs.example/openrtb2/auction"
    timeout_ms = 1500

    [adserver]
    provider = "adserver_mock"

    [adserver.adserver_mock]
    endpoint = "http://127.0.0.1:6767/adserver/mediate"

An integration builder registers an implementation, and a builder that
supplies only implementations is not a page integration a deployment can
name. `openrtb`, `prebid_server`, `aps` and `adserver_mock` register that
way, so core names no vendor and a vendor's own crate registers on the
same terms.

The shared `provider_table` module carries the rules every provider type
follows, so each type reports the same faults the same way. A table its
type's `provider` does not select refuses startup rather than sitting
unread. An implementation this build does not have refuses startup and
the message lists the ones it does have. Every implementation rejects a
setting it does not know, so a misspelt key fails rather than being
dropped. A demand or ad server endpoint must be HTTPS, or plain HTTP to
a loopback address, so a local test stack runs without certificates and
nothing leaves the machine unencrypted.

`auction::demand` is the seam itself. An implementation states the
standard `OpenRTB` field choices it makes, adds its own extensions,
adjusts its outbound headers and reads its own responses. The shared
driver builds every other field, enforces privacy, and applies the
notification policy and routing diagnostics the same way for all of
them, so an implementation can leave data out of what enforcement
approved and can never add data enforcement removed.

APS stops being an integration. It is selected in `[demand]`, its
`rendering_mode` sits in its own demand table, and the renderer
registers from the plan. Two APS sources that disagree on the rendering
mode refuse startup, because one page can be rendered only one way.

The word "mediator" is replaced by "ad server" throughout. The debug
option `include_mediator_response` becomes `include_adserver_response`
and the response metadata strategy `parallel_mediation` becomes
`parallel_adserver`.

Breaking changes an operator sees, each refused at startup with a
message naming the new home:

  `[auction.providers.<id>]`  ->  `[demand] provider` and `[demand.<name>]`
  `protocol` and `profile`    ->  `implementation`
  `profile_config` fields     ->  the settings flat in the table
  `[auction] mediator`        ->  `[adserver] provider`
  `[integrations.aps]`        ->  the `aps` demand table
  `[integrations.adserver_mock]` -> `[adserver.adserver_mock]`
Every guide that showed `[integrations.<id>] enabled = true` now shows the
`[integration] provider` list beside the block, the per-integration `enabled`
rows are gone from the settings tables, and the pages that told an operator to
enable an integration say to name it. The integration guide's seam tables say
that a build function runs only for a module the list names and that the CLI
accepts an id it has never heard of, which the registry then refuses at
startup, so a vendor reads both halves of that gap in one place.

The creative processing page and the repository notes said the server
concatenates the modules of the enabled integrations, and they now say the
modules of the integrations that run.

The CHANGELOG carries the migration the startup error points at, and the two
entries in the same release that named the old table are brought with it.
The per-integration switch is gone, so the word goes with it. The registry
query `integration_enabled` is now `integration_runs` over a
`running_integration_ids` list, `gpt_diagnostics::is_enabled` is `runs`, and
`validate_enabled_integrations` is `validate_integration_blocks`, which is what
its own doc comment already said it does. Nothing outside these files used
either name.

The comments that called a module registered but not enabled now say that
`[integration] provider` does not name it, which is what the errors beside them
already told an operator, and the publisher cache tests name the new table in
two comments and in one assertion message.

The DataDome module doc showed `enabled = true` in its example block, which
startup now refuses, so the example names the integration in
`[integration] provider` instead.

A few comment sentences in the template, the integration test fixture and two
doc comments are joined with linking words rather than with a colon, to match
the house style.
The configuration file spelled the same idea four different ways. An
integration was switched on by `enabled = true` inside its own block, a
demand source by a key in `[auction.providers]`, a permission signal by
a `sources` list, and an Edge Cookie provider by a nested
`[ec.providers.<name>]` block. Someone writing one of these files had
four patterns to learn and no way to tell which fault a mistake would
produce.

The new page `docs/guide/configuration-rules.md` sets out the single
rule, what Trusted Server checks and when, and why one syntax plus
refusal at startup helps the people who run deployments and the people
who write providers. It carries a complete worked example and a table
from the previous layout to the new one.

The rest of the documentation, the example configuration, the
integration fixtures and the template-cache script follow the same
convention, so nothing shows a reader a shape the code no longer
accepts.

Two claims are corrected rather than repeated. `ts config validate` does
not run the Edge Cookie, location, device or permission signal selection
checks, which live in the path only a running instance takes, so the
page says plainly which checks happen at push time and which at startup,
and that a passing validate is not proof that a change to those four
will start. The auction orchestration guide no longer describes an ad
server as a mediator.
Both halves of the provider convention land together, so the
configuration file has one rule rather than two new ones.

The integration half brings `[integration] provider`, which is the only
switch an integration has. The demand half brings `[demand]` and
`[adserver]`, and it takes APS and the ad server mock out of the
integration table, because neither is a page integration. Where the two
disagreed the resolution follows that split: the integration branch owns
`[integration]`, its registry and its documentation pages, and the demand
branch owns the ad server, APS and Prebid Server as implementations.

Core reserves only the Prebid id now, since APS and the ad server mock
reach the registry through the compiled plan rather than through
`[integration] provider`, and the APS renderer registers from the demand
sources the plan selected.
The adapter test still built an `[auction.providers]` entry, which the
provider convention removed. It now selects an APS demand source the way
a deployment does, so the test proves the renderer registers from the
compiled plan rather than from a table that no longer exists.
Every adapter's startup test still built `[auction.providers]` entries,
which the provider convention removed. Each now selects its sources the
way a deployment does, so the tests still prove what they were written
to prove: that the APS renderer route is registered from the compiled
plan, and that Cloudflare and Spin refuse a plan with two demand sources
because their HTTP clients run one request at a time.

The Spin adapter gains serde_json as a development dependency, which is
what writing a settings table in a test needs.
Brings the Edge Cookie provider block layout, where each provider has its
own [ec.<name>] table and an optional implementation = "<id>" line, under
the host-signal provider rename this branch already carried. Both changes
are the same decision applied to different parts of one section, so the
host-signal provider is now selected with [ec] provider = "host_signals"
and configured in [ec.host_signals], not [ec.providers.host_signals].

Four files conflicted.

crates/trusted-server-core/src/settings.rs. split/1 replaced the typed
EcProviders struct with EcProviderBlocks, a map of the blocks written
under [ec], in which only the hmac implementation keeps typed settings
and everything else is held as the raw values an adapter reads. This
branch had added host_signals as a second typed built-in beside hmac, so
the resolution gives it the same standing in the new model rather than
demoting it to raw values. EcProviderSettings gains a HostSignals
variant, EcProviderBlock gains host_signals_settings, EcProviderBlocks
gains host_signals_blocks beside hmac_blocks, read_provider_block reads a
host_signals block into the typed config, and From<HostSignalsProviderConfig>
builds the block. The block-required check in validate_provider_selection
now covers both implementations built into core, because both take a
passphrase. The placeholder-secret report names the block the operator
wrote, as it already did for hmac. The refusal of the old host-signals
spelling is kept and its message now sends the operator to
[ec.host_signals]. That refusal moved to the top of
validate_provider_selection and now covers a block left under the old
name as well as the selector, because split/1's snake_case rule runs over
the block names first and would otherwise refuse host-signals with the
general rule instead of the message naming the spelling to write. The
tests of both branches are kept, with this branch's host-signal test
rewritten onto the new block layout.

crates/trusted-server-core/src/ec/provider.rs. Both branches edited the
resolution arms and the module documentation. build_provider keeps the
host_signals parameter this branch added, because the host-signal
provider needs a service only some hosts supply, and the host-signal arm
of resolve_named_provider now reads its settings out of the block the
selector names rather than a fixed field. BUILTIN_PROVIDER_KEYS keeps
both entries under split/1's wording, which calls them implementation
ids. The documentation keeps split/1's [ec.<name>] spelling together with
this branch's statement that the host-signal provider is built per
request.

crates/trusted-server-core/src/config.rs. The registered secret path for
the host-signal passphrase drops the removed providers segment and
becomes ec.host_signals.passphrase. Push-time validation reads the
host-signal blocks the same way it reads the hmac ones, and the reader
that finds a built-in provider configured under a label of the operator's
choosing now looks for either implementation, so a labeled host-signal
block resolves its passphrase too.

crates/trusted-server-core/src/config_payload.rs. Both branches added a
key to the test secret store. Both are kept, being separate keys serving
separate tests.

Two follow-on fixes outside the conflicts. The tests this branch had
written against the removed EcProviders type now go through a new
select_host_signals_provider test helper beside select_hmac_provider, and
the build_provider calls split/1 had reduced to two arguments take the
host_signals argument again.

Tests. cargo fmt --all -- --check is clean and cargo clippy -p
trusted-server-core --all-targets --all-features -- -D warnings passes.
The core suite passes 2,782 tests and 5 doc-tests with 4 ignored. The
Axum adapter passes 43 tests across its four binaries, cargo clippy-axum
is clean, and cargo check -p trusted-server-adapter-fastly --target
wasm32-wasip1 builds.
…it/2

Brings up the Edge Cookie provider block layout from split/1, where each
provider has its own [ec.<name>] table, together with split/2's rename of
the host-signal provider to host_signals. Both are the same decision this
branch applies to permission signals, which is that every provider name
an operator types is snake_case and a provider type is a top-level table
whose provider key selects what runs.

Nothing conflicted. The permission signal work this branch carries sits
in its own section and its own module, so the Edge Cookie section
changes merged alongside it.

One fix the merge needed. A test of the jurisdiction acknowledgment built
its stateless case by deleting the hmac provider block from the test
configuration by text, and it still named that block [ec.providers.hmac].
Under the new layout the block is [ec.hmac], so the deletion would have
matched nothing and left a provider block configured with no selector,
which the new layout rejects. The test now names the block it means.

Tests. cargo fmt --all -- --check is clean and cargo clippy -p
trusted-server-core --all-targets --all-features -- -D warnings passes.
The core suite passes 2,874 tests and 8 doc-tests with 4 ignored. The
Axum adapter passes 64 tests across its five binaries, cargo clippy-axum
is clean, and cargo check -p trusted-server-adapter-fastly --target
wasm32-wasip1 builds.
…rom split/3

Brings up the Edge Cookie provider block layout from split/1, the
host_signals rename from split/2 and the permission signal provider key
from split/3, under the client_fixed rename this branch already carried.
All four are the same decision, so this branch's demonstration provider
now sits in the same layout as the rest.

Two files conflicted.

crates/trusted-server-core/src/ec/provider.rs. Both branches added
constants and resolution arms. The client_fixed constants and its
resolution arm are kept, listed in BUILTIN_PROVIDER_KEYS beside the two
providers that derive an identifier at the edge, under split/3's wording,
which calls these implementation ids rather than names. The arm now
matches on the implementation the selected block names rather than the
selector itself, which is how the other arms read since the block layout
landed, so the demonstration provider can be configured under a label as
any other provider can.

check_named_provider_configuration needed a real decision. This branch
wrote it to answer two questions, which are whether the build compiles a
name in and whether the name needs a settings block, the second by
looking every name up in [ec.providers]. The block layout removed that
lookup and moved the block question back into the settings, where it is
now asked only of the implementations core knows take settings, because a
provider an adapter injects has its block read by that adapter and core
cannot say whether it needs one. Reinstating a block requirement for
every name would undo that, so the function keeps only the question this
branch added it for, which is whether the demonstration provider is
compiled into this build, and the settings ask the block question. The
function's own reasoning still holds for what it keeps, which is that
only the resolution knows what is compiled out.

The refusal of the old client-fixed spelling moved with it into the
settings, beside the refusal of the old host-signals spelling, and both
now run before anything else. They have to, because the block layout
holds every provider name to snake_case and both old spellings break that
rule, so leaving either refusal later meant an operator got the general
rule instead of the message naming the spelling to write. Both refusals
cover a block left behind under the old name as well as the selector,
through one names_retired_provider helper.

crates/trusted-server-core/src/settings.rs. The Ec struct keeps this
branch's resolve_allowed_origins field and loses the providers field the
block layout replaced. resolve_allowed_origins is a fixed key of the [ec]
section, so it joins EC_SECTION_KEYS, without which the section would read
it as a provider block.

Four of this branch's tests followed the new layout. The one that proves
a provider with no block validates now says why the settings no longer
demand one, the old-spelling test builds its configuration through the
[ec] section helper rather than by deleting the hmac block by text, the
missing-block error names [ec.hmac], and the stale-block test builds its
block through the shared test helper.

Tests. cargo fmt --all -- --check is clean and cargo clippy -p
trusted-server-core --all-targets --all-features -- -D warnings passes.
The core suite passes 2,909 tests and 8 doc-tests with 4 ignored. The
Axum adapter passes 64 tests across its five binaries, cargo clippy-axum
is clean, and cargo check -p trusted-server-adapter-fastly --target
wasm32-wasip1 builds.
…guides

Brings up the Edge Cookie provider block layout from split/1, the
host_signals rename from split/2, the permission signal provider key from
split/3 and the client_fixed rename from split/4. All four are the same
decision, which is that every provider name an operator types is
snake_case, a provider type is a top-level table, and a provider key
inside it selects what runs.

Nothing conflicted. The response hook and the documentation set this
branch carries sit beside the sections those renames touched.

The documentation needed the work the clean merge did not do. These
guides were written against the old layout, so they still told operators
to write a table that is now refused and to select providers by spellings
that now stop startup. The configuration guide, the Edge Cookie setup
guide, the error reference, the Fastly guide and the key rotation guide
all move from [ec.providers.hmac] to [ec.hmac], and the environment
override for that passphrase becomes
TRUSTED_SERVER__EC__HMAC__PASSPHRASE, which is the path the settings now
hold it at.

The configuration guide's [ec] reference needed more than a rename. Its
rule that every selection must have a matching block is no longer what
the code does, because a provider has a block only when it has settings,
so it now says which providers need one and which do not, and it explains
the implementation key that lets a block carry a label of the operator's
choosing. Its provider list and its validation notes follow the
snake_case names, and the resolve_allowed_origins key this stack added to
the [ec] section is documented alongside the rest.

The Edge Cookie guide named the host-signal and demonstration providers
by their old spellings throughout, including in a diagram label, and now
names them host_signals and client_fixed. The cargo feature that compiles
the demonstration provider in keeps the name client-fixed-demo, because a
cargo feature name is not a configuration value.

The example configuration's commented-out host-signal block carried both
the old table and the old name, and is now [ec.host_signals] selected by
provider = "host_signals".

Tests. cargo fmt --all -- --check is clean and cargo clippy -p
trusted-server-core --all-targets --all-features -- -D warnings passes.
The core suite passes 2,909 tests and 8 doc-tests with 4 ignored. The
Axum adapter passes 64 tests across its five binaries, cargo clippy-axum
is clean, and cargo check -p trusted-server-adapter-fastly --target
wasm32-wasip1 builds.
This brings the rest of the stack onto the seam branch, so every
provider type in the file now follows one rule rather than the demand
and ad server pair following it alone.

Where the two sides disagreed, each kept what it owns. The Edge Cookie
side owns `[ec.<name>]`, the optional `implementation` line and the
label-aware passphrase handling, so the secret paths become
`ec.hmac.passphrase` and `ec.host_signals.passphrase` and a block written
under a label of the operator's choosing has its key name read out of
the configuration rather than from a fixed path. The seam side owns
`[demand]`, `[adserver]` and `[integration]`, the ad server vocabulary
and the corrected note that `ts config validate` does not run the
provider selection checks.

The adapters keep this branch's field name for the Edge Cookie provider
they resolve per request, and take split/5's corrected selector name for
the permission signals, which is `provider` rather than `sources`.

Three test fixtures still wrote `[ec.providers.<name>]`, which startup
now refuses, so they are restated in the new shape.
Four sentences in the guides joined their clauses with a semicolon, and
one table row grew past its column. They now read as one sentence each,
and the table is realigned, so the pages follow the house style the rest
of the guide follows.
The seam promised that an implementation decides only its field policy,
its own extensions, its outbound headers and how its responses are read,
while the driver builds every standard field and enforces privacy. The
two request hooks did not hold that promise structurally. augment_request
received the whole OpenRTB request, so an implementation could rewrite
any standard field the driver had built, and prepare_outbound received
the whole outbound request, so it could replace the body.

augment_request now receives the extension objects and nothing else, one
for the request and one for each impression, each impression beside the
routed slot it was built from. That removes the zip by position that
Prebid Server used to pair impressions with slots, which a slot with no
valid format could have misaligned. prepare_outbound now receives the
outbound headers alone, and the driver captures the headers as they will
be sent, after the implementation has added its own.

The driver also refuses a request whose extension claims
ext.trusted_server, because that object is the driver's signature and
identity block and no implementation may write it. The plain OpenRTB
implementation already refused it in its static extensions at startup,
and now the driver holds the line for every implementation.
… the code does

The configuration rules page promised that every provider rejects a
setting it does not know, and nine of the thirteen page integrations did
not. A misspelled key in a Didomi, Google Tag Manager, GPT, asset proxy,
Lockr, Permutive, Sourcepoint, Testlight or Next.js block was ignored.
Each of their settings types now refuses unknown keys, and a test walks
every built-in page integration to prove a planted key fails deploy
validation naming the integration and the key.

The rules page and the CHANGELOG now say only what the code does. The
push-time list names the refusal of an unselected block, of an enabled
key left behind and of the removed [integrations] table, rather than a
typo in a switched-off block being caught, which under the new selector
cannot exist. The startup list says that a module declaring an identity
or device provider no type selects is logged, which is what the registry
does, rather than reported. The two overlapping CHANGELOG entries for the
convention and for the integration selector become one, carrying what
the second said that the first did not.

The rest is the sweep before these changes go public: the mediator
rename had left "a ad server", "ad server=none" and a mangled function
name in comments, an environment example carried a stray separator, the
Prebid table header in the example configuration had lost its line
break, and the getting-started guide still sent readers to the
[auction.providers] table this stack removed. Em dashes and
clause-joining marks in lines this stack added are gone, the loopback
endpoint test uses a documentation address rather than a private one,
and the documentation tables are realigned.
# Conflicts:
#	docs/guide/configuration.md
The fixture selecting a vendor provider this adapter does not inject writes its settings in [ec.acme], as every provider block is written since the tables moved under [ec], and the comment above it still named the removed [ec.providers.<key>] table. The comment now says what the fixture does, in the words the other branches of this series already use.
…te's example

Two audit tests counted the string [[integration.js_asset_proxy.assets]]
in the draft configuration to prove how many asset entries the audit
wrote. The draft is built on the example configuration, which now
documents that array as a commented example, so the count included the
example and the two tests failed on Linux, which is where CI runs them.

The first test now reads the draft as TOML and counts the entries in the
array the audit wrote, which is what it meant to check, and the second
asserts that no uncommented line opens that array. Both stay silent
about the example, because a commented line configures nothing.
… writes

The environment overlay tests built their project by adding an
[auction.providers.pbs-main] table and an example-bidder route naming it
to the integration fixture, which is the layout this series removed.
Every test that pushed or validated that project failed on Linux, where
CI runs the CLI job, because [auction.providers] is refused with the
move spelled out and the hyphenated name is not snake_case.

The fixture already carries [demand.pbs_main] with its endpoint and the
[auction.bidders.example-bidder] route that names pbs_main, so the
project no longer adds a second copy under the old names. The endpoint
overlay now addresses TRUSTED_SERVER__DEMAND__PBS_MAIN__ENDPOINT and
the pushed envelope is read back from the demand table, and the legacy
provider list test asserts the refusal names [demand] provider, which is
where the message sends an operator, rather than the CHANGELOG, which
it does not mention.
Three fixtures in the Fastly adapter's own tests still described the
auction the old way, two with an [auction.providers.prebid] table and one
with an empty providers map, which this series refuses with the move
spelled out. Those tests run only under Viceroy on wasm32-wasip1, so no
native gate reached them and the first of them crashed the CI job.

The two Prebid fixtures now select a demand source named prebid whose
implementation is prebid_server, with the same endpoint and timeout, and
the empty map is gone because an empty demand list is the default. All
192 tests in the adapter's binary pass under Viceroy 0.17.0.
Brings in the eight commits main gained since this branch was cut, of which
three touch the same code as the provider seam.

The parser-aware body hold gives every adapter a second entry point,
`build_state_with_services`, so a caller can supply the `RuntimeServices`
each request uses. The composition-root check the seam added now runs in
that function rather than in `build_state_with_settings`, and it is given
whatever provider those supplied services already carry instead of always
`None`, so a caller that resolved one is not made to resolve it twice.

On Cloudflare and Spin the state keeps both the provider this branch
resolves once at start-up and the services main lets a caller supply. The
free function this branch added is gone and its one job, handing the
resolved provider to every request, moved into `services_for_request`,
which is the method main introduced for the same purpose.

The core README this branch corrected one line of has been rewritten
wholesale by the documentation refresh, and the section that line was in no
longer exists, so main's version is taken as it stands.
Carries up the resolutions from split/1, where main's second entry point,
`build_state_with_services`, meets the composition-root provider check.

This branch had already given that check and `build_reusable_provider` a
`host_signals` argument, so each call now passes `None` for host signals and
whatever provider a caller's supplied services already carry. The free
function that handed the resolved provider to every request is gone on both
adapters and its job sits in `services_for_request`, the method main added,
which passes the settings this branch made `build_runtime_services` take.
Carries up the resolutions from split/2 and settles what main's own changes
mean for the permission model.

The adapters keep both the permission signal providers this branch resolves
at start-up and the services main lets a caller supply, and
`services_for_request` passes the providers to `build_runtime_services`, so
a request served from supplied services and one served from the request
context see the same schemes.

Main added tests for `allows_ec_creation`, `has_explicit_ec_withdrawal` and
`gate_eids_by_consent`, all three of which this branch replaced with the
permission model, so those tests name functions that no longer exist and
have not been taken. The behaviour the new one covers, stripping every EID
when personalised advertising is not permitted, is already asserted here by
`gate_eids_strips_eids_when_a_required_permission_is_unset`, which reaches
it through the resolved permissions rather than through the consent context.

The no-op HTML post processor in the registry tests goes with
`IntegrationHtmlPostProcessor`, the trait main removed in favour of
per-document stream processors.
Carries up the resolutions from split/3. Nothing on this branch touches the
code main changed, so the merge needed no decision of its own.
…once

A route whose parameter the request's permissions withhold answers 403 as
before. A value absent where the route runs, or needing permissions that are
not resolved there, is a fault in how the module was written or placed, and
answers 500 rather than 403.

The injection rule is stated in the module_context rustdoc and the trait
methods link to it. The route helpers' repeated doc sentence, three repeats of
the Edge Cookie snapshot comment and the guide's copy of the parameter table
are removed. A misplaced test helper doc is moved back to its function, and
the device-signal gate's limit for a module the adapter builds is stated.
The deploy check of the inspect patterns now sees every secret leaf the settings load resolves, including a labeled Edge Cookie block's passphrase and a module's declared secrets. A show pattern naming one of them is refused at ts config validate as it is at startup. A test covers both kinds. Comments in the inspection code now state each rule once, and the API reference says the data page answers 403 on a plain http host other than localhost.
Next.js implements TextHandler on a per-document struct for each of its two script rewriters, and the private ScriptRewriter trait is gone. The registry's middleware messages say module. The fetch and serve rule is stated once, on the MiddlewarePhase variants, and the remaining comments, the changelog and the Next.js guide drop hook and rewriter wording.
An operator's response_headers replaced headers of the same name as the
response was finalized, so a site-wide Content-Security-Policy removed the
sandbox from /_ts/data and a page that opened it could read the record. The
data page now marks its headers as required, and core sets them again after
the operator's headers on every adapter.
# Conflicts:
#	crates/trusted-server-core/src/auction/endpoints.rs
#	crates/trusted-server-core/src/ec/finalize.rs
#	crates/trusted-server-core/src/test_support.rs
# Conflicts:
#	crates/trusted-server-adapter-fastly/src/main.rs
#	crates/trusted-server-core/src/test_support.rs
main's new EID persistence gate is the permission model's sharing check. EIDs are written to the identity graph only when the storage and personalised-ad permissions are both set, the pair that already gates the bid request, so allows_eid_persistence and its parity test are removed and the tests that described Purpose 4 denial state the permissions instead.
main's rule that partner EIDs are written to the identity graph only with consent for personalised ads is stated in the guides as the storage and personalised-ad permissions, the pair the permission model already uses for sharing.
…d run the Cloudflare auction test with no demand modules

The build script's TRUSTED_SERVER__GIT_VERSION reaches test processes and the strict loader refused it as an unknown field. Cloudflare runs one demand source at a time, so the fixture's two demand modules stopped worker startup.
@jwrosewell

Copy link
Copy Markdown
Collaborator

Moved into this repository as #1351, one branch of the stack, so the review continues there. This pull request is closed in its favor and its comments stay here for the record.

@jwrosewell jwrosewell closed this Oct 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A module changes every page, in the order the modules were registered, through four hook traits

2 participants