Skip to content

Let a middleware run on a media type other than HTML - #1360

Open
jwrosewell wants to merge 36 commits into
feature/18-inmobifrom
feature/19-media-types
Open

jwrosewell wants to merge 36 commits into
feature/18-inmobifrom
feature/19-media-types

Conversation

@jwrosewell

@jwrosewell jwrosewell commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Changes

Area Change
crates/trusted-server-core/src 5 files, 1152 lines added, 49 removed
docs/guide 2 files, 75 lines added, 2 removed
crates/cmp/inmobi 1 file, 8 lines added, 4 removed
CHANGELOG.md 1 file, 1 line added, 0 removed

Closes

Closes #1277

Spec

Spec: middleware-media-types

Test plan

CI on this pull request at 18d9ff9 passed all 32 checks.

  • cargo test-fastly && cargo test-axum
  • cargo clippy-fastly && cargo clippy-axum
  • cargo fmt --all -- --check
  • JS tests: cd crates/trusted-server-js/lib && npx vitest run
  • JS format: cd crates/trusted-server-js/lib && npm run format
  • Docs format: cd docs && npm run format
  • WASM build: cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1
  • Manual testing via fastly compute serve
  • Other: the full workflow set of this repository, run on this pull request at 18d9ff9

Checklist

  • Changes follow AGENTS.md conventions
  • No unwrap() in production code (clippy denies unwrap_used in CI)
  • Uses log macros as AGENTS.md asks (not println!)
  • New code has tests
  • No secrets or credentials committed

Stacks on the InMobi Choice consent prompt (#1359).

What this pull request does

[[fetch]]
media_type = "text/plain"
path = "/ads.txt"
middleware = ["example.sellers"]

An entry may cover any media type core streams as text: text/html, another text/ type other than text/x-component, application/javascript or application/json, written as a bare type/subtype in lower case. A response takes the entry for the type and subtype of its content type, without the parameters, so text/plain; charset=utf-8 takes the entry for text/plain. An image, a length prefixed stream or a response that declares a charset other than UTF-8 passes through untouched and no entry can cover it.

pub trait Middleware: Send + Sync {
    fn middleware_id(&self) -> &'static str;
    fn handles_media_type(&self, media_type: &str) -> bool {
        media_type == HTML_MEDIA_TYPE
    }
    fn phases(&self) -> &[MiddlewarePhase];
    fn create(&self, context: &MiddlewareContext<'_>) -> MiddlewareAction;
}
  • A middleware says what it works on. HTML alone unless it says otherwise, so every module in the stack keeps its behavior with no change to its code. An entry naming a middleware under a media type it does not handle is refused when the registry is built, naming the entry and the middleware.
  • A chain carries its media type. On one other than HTML its plan refuses an action that decides about HTML, being head markup or an element or text handler, by the middleware's name. A stream processor is the one thing that applies to any body.
  • Core runs the stream processors on the body after it has moved the origin's addresses in it, the fetch entry's and then the serve entry's. None can be stored, because the template cache refuses anything that is not HTML.
Configuration Refused with
An entry for image/png, image/svg+xml, text/x-component, */*, TEXT/HTML, a type with parameters, an empty subtype or whitespace "is not a media type a middleware runs on", and the types to write
An entry naming a middleware under a media type it does not handle The entry, the media type and the middleware's name
A middleware's head markup or handler on a body that is not HTML The middleware's name and the media type, when the chain is planned

Why

Every middleware in the stack changes HTML, and [[fetch]] and [[serve]] refused any other media type. A publisher's ads.txt, robots.txt beside it, a script or a stylesheet are text that passes through the same path, and a module that has something to add to one of them had no place to do it.

How it was verified

Not exercised: no module of the stack changes a body that is not HTML yet. The fixture middleware in core's tests is the one that does, through the publisher's own path.

References #1277.

jwrosewell and others added 11 commits October 10, 2026 03:28
An entry may cover any media type core streams as text: `text/html`,
another `text/` type other than `text/x-component`, whose rows are length
prefixed, `application/javascript`, `application/json` or `image/svg+xml`,
written as a bare `type/subtype` in lower case. A response is matched by the
type and subtype of its content type, without the parameters.

A middleware says which media types it works on with `handles_media_type`,
HTML alone unless it says otherwise, so every module keeps its behavior
with no change to its code. An entry naming a middleware under a media type
it does not handle is refused when the registry is built, naming the entry
and the middleware. A chain carries its media type, and on one other than
HTML its plan refuses an action that decides about HTML, being head markup
or an element or text handler, by the middleware's name, because those are
decisions about HTML and a stream processor is the one thing that applies
to any body.

The publisher runs the covering entries' stream processors on a body of
such a media type after it has moved the origin's addresses in it: the
fetch entry's on what is stored, and the serve entry's on each reader's
copy, whether that copy is assembled from the store or fetched for the
request, as on a page.

Core's test fixture gains a middleware that works on `text/plain` and adds
a line at the end of a body, and the tests cover the entries accepted and
refused, the media type of a content type, a chain refusing an HTML
decision on text and taking a stream, the registry refusing a middleware
under a media type it does not handle in either direction, and a text body
through the publisher with and without a covering entry.
The configuration guide says which media types an entry may cover, how a
response is matched, what runs on a body that is not HTML and what is
refused, with an entry for a text file as the example. The integration
guide shows `handles_media_type` on a middleware that works on
`text/plain`. The changelog records the change.
# Conflicts:
#	CHANGELOG.md
#	crates/trusted-server-core/src/middleware.rs
#	crates/trusted-server-core/src/publisher.rs
An entry may no longer cover image/svg+xml, which core passes through before any middleware runs, and an entry whose media type has an empty subtype or whitespace is refused. A response that declares a charset other than UTF-8 is handed to no middleware, so a UTF-8 append cannot corrupt it. The stored-response branches for a body that is not HTML are gone, because the template cache stores HTML alone, so both phases run as the response is fetched. TextDocument is TextResponse, and the guide and comments say each rule once.
# Conflicts:
#	crates/trusted-server-core/src/integrations/registry.rs
#	crates/trusted-server-core/src/middleware.rs
#	crates/trusted-server-core/src/publisher.rs
The entries covering a text response ran their middleware's stream
processors and dropped the headers a serve middleware wrote. The headers
now come back with the processors and are written on the response on both
paths, as they are for a page.
…vate

A page for a request that modules left state on is sent `private, no-store`
and loses its validators, so no shared cache replays it to another reader.
A text response a `[[serve]]` entry covers is handed the same state, but it
kept the origin's caching directives, so a shared cache could replay a body
shaped for one reader to the next.

Both finalizers now apply the same policy to such a response, once the
serve middleware's own headers are written, so none of them can undo it.
`text_entry_media_type` holds the rule for which responses the entries run
on, so the finalizers and the body processor decide alike.

The test sends a request a module left state on for a text file through the
publisher path and both finalizers, and checks that a request with no state
keeps the origin's `Cache-Control`.
Every covered text response ran the entries' middleware whatever its
status, so a line added to a `206` slice left its `Content-Range` untrue,
and an error page was changed as if it were the file. Only a `200` carries
the whole file, so the entries' middleware now run on that alone. Core
still moves the origin's addresses in every text response, as before.

The status travels with the response's other details in the parameters of
the body processing. The test sends a `200`, a `206`, a `404` and a `500`
text response through the publisher path and both finalizers.
A body a middleware changes kept the origin's `ETag` and `Last-Modified`,
so a reader asking again with them got a `304` from the origin while the
file was unchanged, and kept a body without what the middleware would add
now. The headers go out before a streamed body is processed, so no new
validator can be computed, and both finalizers remove the two headers from
a response an entry covers.

The test checks both finalizers, and that a body no entry covers keeps both
headers.
The changelog entry, the configuration guide and the integration guide now
say that a middleware runs on a text body only for a `200`, that a response
an entry covers loses the origin's validators, and that one a `[[serve]]`
entry covers is sent `private, no-store` when a module left state on the
request.
@jwrosewell
jwrosewell removed this pull request from stack #1408 October 18, 2026 20:15
@jwrosewell
jwrosewell added this pull request to stack #1418 October 19, 2026 12:18
@jwrosewell
jwrosewell removed this pull request from stack #1418 October 19, 2026 16:30
@jwrosewell
jwrosewell added this pull request to stack #1421 October 19, 2026 17:38
@jwrosewell
jwrosewell removed this pull request from stack #1421 October 19, 2026 18:29
@jwrosewell
jwrosewell added this pull request to stack #1422 October 19, 2026 19:20
@jwrosewell
jwrosewell marked this pull request as ready for review October 19, 2026 20:23
@jwrosewell
jwrosewell removed this pull request from stack #1422 October 28, 2026 11:47

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A middleware can change HTML and nothing else, so a module has no place to change a text file

2 participants