diff --git a/CHANGELOG.md b/CHANGELOG.md index 9a647c13f9..f0015c5cd4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed +- **Breaking for crates outside core:** the identifiers a request carries come from identifier modules. Core declares `IdentifierModule` in `trusted_server_core::ec::identifiers`, a registration offers one with `IntegrationRegistrationBuilder::with_identifier_module`, and core asks every running module once per request, when it reads the Edge Cookie state, then syncs the answers to the identity graph and carries them to the auction as it did the `ts-eids` cookie. The Prebid module owns that cookie, its two shapes and its size limit, and is the first identifier module. `COOKIE_EIDS` is removed from core, `ec_finalize_response` no longer takes the cookie's value, `EcContext::module_eids` holds what the modules learned, and `RuntimeServices` carries the modules through `with_identifier_modules`. A `template_cache_key_cookies` entry naming any cookie under the public prefix is refused, where the Edge Cookie and the EIDs cookie were refused by name. A deployment's settings do not change. - **Breaking:** The React Server Components Flight rewriter is the middleware `framework.nextjs.flight` of the Next.js module, which runs on a `text/x-component` response where a `[[fetch]]` entry names it, where core rewrote every Flight response by media type. A deployment that fronts a Next.js site selects `[framework] module = "nextjs"` and adds the entry, as `trusted-server.example.toml` shows it. With the module selected and no entry, a Flight response passes through as the origin sent it, and an entry may now cover `text/x-component`. A middleware says with `Middleware::moves_origin_address` that it moves the origin's address itself, and core then leaves the addresses in a body of the media types it handles to it, whether or not an entry names it, because a plain replacement in a length prefixed row leaves the old length. **For crates outside core**, `trusted_server_core::rsc_flight` is removed. - **Breaking:** Where content comes from is a source module. `[source]` selects the publisher's, and each `[[proxy.asset_routes]]` entry selects its own in a `source` table of the same shape, with `module` naming the source and its settings in the table beneath. Core supplies `origin`, an HTTP origin, and a module a build links can offer another through `IntegrationRegistrationBuilder::with_source_module`. A source answers for each request where that request's content comes from, and core fetches, caches, rewrites the origin host and follows the redirect rule from that answer. `[source]` is required. The origin settings move into the `origin` table, and a document that still writes one where it was is refused at startup naming the new place. An environment variable that set the address is now `TRUSTED_SERVER__SOURCE__ORIGIN__URL`. The old name is refused where the build reads variables, and `ts config push` ignores it, as it ignores any variable for a key the document does not hold. A path in an asset route's `url` names a folder the target path is added under, where a path was refused, a target path with a `.` or `..` segment under a folder is answered `400`, and a route applies `directory_index` as the publisher does. Stored pages are fetched again after the upgrade, because their keys hold a code worked out from the settings. Where `host_header` is set, the page pipeline and every middleware are now told that `Host` as the page's origin host, where they were told the address's host, so the page's own addresses under that name are rewritten to the request's host. **For crates outside core**, `handle_asset_proxy_request` takes the route's source, which `IntegrationRegistry::route_source` gives, and `Publisher::origin_host`, `origin_host_header` and `origin_query_is_stripped` are replaced by `Settings::origin_source` and the `SourceLocation` a source answers. diff --git a/crates/auction/prebid/src/eids.rs b/crates/auction/prebid/src/eids.rs new file mode 100644 index 0000000000..1cfe4f6119 --- /dev/null +++ b/crates/auction/prebid/src/eids.rs @@ -0,0 +1,396 @@ +//! The cookie the Prebid page script persists the browser's identifiers in, +//! and the identifier module that reads it. +//! +//! The script writes `-eids`, such as `ts-eids`, holding a +//! base64 JSON array of `OpenRTB`-style entries (`{source, uids: [...]}`) from +//! Prebid's user ID modules, trimmed by the script to fit its own size cap. +//! The earlier flattened shape (`{source, id, atype}` per entry) is still +//! read, for a cookie written before the script changed. Core is handed what +//! the cookie holds through [`IdentifierModule`] and names the cookie +//! nowhere. + +use base64::{Engine as _, engine::general_purpose::STANDARD as BASE64}; +use serde::Deserialize; +use serde_json::Value as JsonValue; + +use trusted_server_core::ec::identifiers::IdentifierModule; +use trusted_server_core::evidence::RequestInfo; +use trusted_server_core::module_context::{ModuleCall, Withheld}; +use trusted_server_core::openrtb::{Eid, Uid}; +use trusted_server_core::platform::RuntimeServices; + +/// The cookie the script writes, named after the public prefix with +/// `PublicPrefix::cookie_name`, so `ts-eids` under core's own prefix. +pub const EID_COOKIE: &str = "eids"; + +/// Maximum raw cookie size accepted before base64 decode. +const MAX_EIDS_COOKIE_BYTES: usize = 8 * 1024; + +/// Reads the identifiers the script persisted for the request. +#[derive(Debug, Default, Clone, Copy)] +pub struct PrebidEids; + +impl PrebidEids { + /// What the cookie holds, or nothing when the request carries none or + /// the value cannot be read. + fn read(&self, evidence: &dyn RequestInfo, services: &RuntimeServices) -> Vec { + let name = services.public_prefix().cookie_name(EID_COOKIE); + let Some(value) = evidence.cookie(&name) else { + return Vec::new(); + }; + match parse_eids_cookie(value) { + Ok(eids) => eids, + Err(error) => { + log::trace!("Prebid EIDs: dropping the `{name}` cookie: {error}"); + Vec::new() + } + } + } +} + +impl IdentifierModule for PrebidEids { + fn id(&self) -> &'static str { + super::MODULE + } + + fn identifiers(&self, call: ModuleCall<'_>) -> Result, Withheld> { + call.inject(self, Self::read) + } +} + +/// Legacy flattened cookie entry. +#[derive(Debug, Deserialize)] +struct LegacyCookieEid { + source: String, + id: String, + atype: i32, +} + +/// `OpenRTB`-style cookie entry. +#[derive(Debug, Deserialize)] +struct StructuredCookieEid { + source: String, + #[serde(default)] + uids: Vec, +} + +#[derive(Debug, Deserialize)] +struct StructuredCookieUid { + id: String, + #[serde(default)] + atype: Option, + #[serde(default)] + ext: Option, +} + +enum DecodedCookieEids { + Legacy(Vec), + Structured(Vec), +} + +/// Parses the cookie's value into `OpenRTB`-style `Eid` entries, in either of +/// the shapes the script has written. +/// +/// # Errors +/// +/// Returns an error when the cookie exceeds the raw size limit, is not valid +/// base64, or does not contain either supported JSON payload shape. +pub fn parse_eids_cookie(cookie_value: &str) -> Result, String> { + decode_eids_cookie(cookie_value).map(DecodedCookieEids::into_openrtb) +} + +fn decode_eids_cookie(cookie_value: &str) -> Result { + if eids_cookie_exceeds_size_limit(cookie_value) { + return Err(format!("cookie too large ({} bytes)", cookie_value.len())); + } + + let bytes = BASE64 + .decode(cookie_value) + .map_err(|e| format!("base64 decode failed: {e}"))?; + + if let Ok(eids) = serde_json::from_slice::>(&bytes) { + return Ok(DecodedCookieEids::Legacy(eids)); + } + + let structured = serde_json::from_slice::>(&bytes) + .map_err(|e| format!("JSON parse failed: {e}"))?; + Ok(DecodedCookieEids::Structured(structured)) +} + +impl DecodedCookieEids { + fn into_openrtb(self) -> Vec { + match self { + Self::Legacy(entries) => legacy_cookie_eids_to_openrtb(entries), + Self::Structured(entries) => structured_cookie_eids_to_openrtb(entries), + } + } +} + +fn eids_cookie_exceeds_size_limit(cookie_value: &str) -> bool { + cookie_value.len() > MAX_EIDS_COOKIE_BYTES +} + +fn structured_cookie_eids_to_openrtb(entries: Vec) -> Vec { + let mut eids = Vec::new(); + + for entry in entries { + if entry.source.is_empty() { + continue; + } + + let uids: Vec<_> = entry + .uids + .into_iter() + .filter_map(structured_cookie_uid_to_openrtb) + .collect(); + if uids.is_empty() { + continue; + } + + eids.push(Eid { + source: entry.source, + uids, + }); + } + + eids +} + +fn structured_cookie_uid_to_openrtb(uid: StructuredCookieUid) -> Option { + if uid.id.is_empty() { + return None; + } + + let atype = uid.atype.filter(|atype| *atype >= 0); + let ext = match uid.ext { + Some(JsonValue::Object(_)) => uid.ext, + _ => None, + }; + + Some(Uid { + id: uid.id, + atype, + ext, + }) +} + +fn legacy_cookie_eids_to_openrtb(entries: Vec) -> Vec { + entries + .into_iter() + .filter(|entry| !entry.source.is_empty() && !entry.id.is_empty()) + .map(|entry| Eid { + source: entry.source, + uids: vec![Uid { + id: entry.id, + atype: (entry.atype >= 0).then_some(entry.atype), + ext: None, + }], + }) + .collect() +} + +#[cfg(test)] +mod tests { + use super::*; + + use http::HeaderMap; + use serde_json::json; + + use trusted_server_core::evidence::OwnedRequestInfo; + use trusted_server_core::module_context::ModuleContext; + use trusted_server_core::module_context::test_support::request; + use trusted_server_core::permissions::PermissionSet; + use trusted_server_core::platform::test_support::noop_services; + + fn encode_json(value: &serde_json::Value) -> String { + BASE64.encode(serde_json::to_vec(value).expect("should serialize EID JSON")) + } + + fn with_cookie_header(header: &str) -> OwnedRequestInfo { + let mut headers = HeaderMap::new(); + headers.insert( + "cookie", + header.parse().expect("should build a cookie header"), + ); + OwnedRequestInfo::new(String::new(), headers) + } + + /// What the module answers for a request carrying `cookie_header`, under + /// core's own prefix. + fn learned(cookie_header: &str) -> Vec { + let evidence = with_cookie_header(cookie_header); + let services = noop_services(); + let context = ModuleContext::new(request("/")) + .with_evidence(&evidence) + .with_services(&services); + PrebidEids + .identifiers(context.call(PrebidEids.id(), PermissionSet::none())) + .expect("the call should carry the evidence and services the module names") + } + + #[test] + fn the_module_reads_the_cookie_under_the_public_prefix() { + let encoded = encode_json(&json!([ + {"source": "id5-sync.com", "uids": [{"id": "ID5_abc", "atype": 1}]} + ])); + + let eids = learned(&format!("other=1; ts-eids={encoded}; another=2")); + + assert_eq!(eids.len(), 1, "should read the cookie named by the prefix"); + assert_eq!(eids[0].source, "id5-sync.com"); + assert_eq!(eids[0].uids[0].id, "ID5_abc"); + } + + #[test] + fn the_module_learns_nothing_from_a_request_without_the_cookie() { + assert!( + learned("other=1").is_empty(), + "a request without the cookie carries no identifiers" + ); + } + + #[test] + fn the_module_learns_nothing_from_a_cookie_it_cannot_read() { + assert!( + learned("ts-eids=not-valid-base64!!!").is_empty(), + "an unreadable cookie should be dropped, not refused" + ); + } + + #[test] + fn the_module_is_withheld_without_the_evidence_it_names() { + let withheld = PrebidEids + .identifiers(ModuleCall::empty()) + .expect_err("a call carrying no evidence should withhold it"); + + assert_eq!(withheld.value(), "request evidence"); + } + + #[test] + fn parse_eids_cookie_parses_legacy_flat_payload() { + let eids = vec![ + json!({"source": "id5-sync.com", "id": "ID5_abc", "atype": 1}), + json!({"source": "liveramp.com", "id": "LR_xyz", "atype": 3}), + json!({"source": "google.com", "id": "pair-id", "atype": 571187}), + ]; + let encoded = BASE64.encode(serde_json::to_vec(&eids).expect("should serialize")); + + let decoded = parse_eids_cookie(&encoded).expect("should decode valid payload"); + assert_eq!(decoded.len(), 3, "should parse all EIDs"); + assert_eq!(decoded[0].source, "id5-sync.com"); + assert_eq!(decoded[0].uids[0].id, "ID5_abc"); + assert_eq!(decoded[1].source, "liveramp.com"); + assert_eq!(decoded[1].uids[0].id, "LR_xyz"); + assert_eq!( + decoded[2].source, "google.com", + "should preserve PAIR source" + ); + assert_eq!( + decoded[2].uids[0].atype, + Some(571187), + "should preserve PAIR vendor-specific atype" + ); + } + + #[test] + fn parse_eids_cookie_parses_structured_payload() { + let eids = vec![json!({ + "source": "sharedid.org", + "uids": [ + {"id": "shared_123", "atype": 3}, + {"id": "shared_456", "ext": {"provider": "example"}}, + {"id": "shared_invalid", "atype": -1} + ] + })]; + let encoded = BASE64.encode(serde_json::to_vec(&eids).expect("should serialize")); + + let decoded = parse_eids_cookie(&encoded).expect("should decode valid payload"); + assert_eq!(decoded.len(), 1, "should parse one structured EID entry"); + assert_eq!(decoded[0].source, "sharedid.org"); + assert_eq!(decoded[0].uids.len(), 3, "should preserve multiple UIDs"); + assert_eq!(decoded[0].uids[0].id, "shared_123"); + assert_eq!(decoded[0].uids[0].atype, Some(3)); + assert_eq!( + decoded[0].uids[1].ext, + Some(json!({"provider": "example"})), + "should preserve UID ext objects" + ); + assert_eq!( + decoded[0].uids[2].atype, None, + "should drop negative atype values" + ); + } + + #[test] + fn parse_eids_cookie_preserves_pair_atype() { + let encoded = encode_json(&json!([ + { + "source": "google.com", + "uids": [{ "id": "pair-id", "atype": 571187 }] + } + ])); + + let decoded = parse_eids_cookie(&encoded).expect("should decode PAIR EID"); + + assert_eq!( + decoded[0].uids[0].atype, + Some(571187), + "should preserve PAIR's vendor-specific atype" + ); + } + + #[test] + fn parse_eids_cookie_skips_entries_with_nothing_to_offer() { + let encoded = encode_json(&json!([ + {"source": "", "uids": [{"id": "no-source", "atype": 1}]}, + {"source": "empty.example", "uids": []}, + {"source": "blank.example", "uids": [{"id": "", "atype": 1}]}, + {"source": "kept.example", "uids": [{"id": "kept", "ext": "not-an-object"}]} + ])); + + let decoded = parse_eids_cookie(&encoded).expect("should decode the payload"); + + assert_eq!( + decoded.len(), + 1, + "should keep only the entry with a source and a uid" + ); + assert_eq!(decoded[0].source, "kept.example"); + assert_eq!( + decoded[0].uids[0].ext, None, + "an ext that is not an object should be dropped" + ); + } + + #[test] + fn parse_eids_cookie_rejects_invalid_base64() { + let result = parse_eids_cookie("not-valid-base64!!!"); + assert!(result.is_err(), "should reject invalid base64"); + } + + #[test] + fn parse_eids_cookie_rejects_invalid_json() { + let encoded = BASE64.encode(b"not json"); + let result = parse_eids_cookie(&encoded); + assert!(result.is_err(), "should reject invalid JSON"); + } + + #[test] + fn eids_cookie_rejects_oversized_payloads() { + let oversized = "x".repeat(MAX_EIDS_COOKIE_BYTES + 1); + let exact_limit = "x".repeat(MAX_EIDS_COOKIE_BYTES); + + assert!( + eids_cookie_exceeds_size_limit(&oversized), + "should reject cookies larger than the raw size cap" + ); + assert!( + !eids_cookie_exceeds_size_limit(&exact_limit), + "should allow cookies exactly at the raw size cap" + ); + assert!( + parse_eids_cookie(&oversized).is_err(), + "an oversized cookie should be refused before decoding" + ); + } +} diff --git a/crates/auction/prebid/src/lib.rs b/crates/auction/prebid/src/lib.rs index c9f70825f4..19f7b061c8 100644 --- a/crates/auction/prebid/src/lib.rs +++ b/crates/auction/prebid/src/lib.rs @@ -4,6 +4,8 @@ //! ``, serves the publisher's Prebid bundle from a first-party route and //! removes the publisher's own Prebid script from the page. The bidders it //! tells the browser to leave to the server are the auction plan's routes. +//! It also reads the cookie its page script persists the browser's +//! identifiers in, and hands core what it holds, see [`eids`]. #![cfg_attr( test, @@ -17,6 +19,8 @@ ) )] +pub mod eids; + use std::collections::HashSet; use std::rc::Rc; use std::sync::{Arc, LazyLock}; @@ -1104,6 +1108,7 @@ pub fn register_for_plan( IntegrationRegistration::builder(PREBID_INTEGRATION_ID) .with_proxy(integration.clone()) .with_middleware(Arc::new(PageChange::new(integration))) + .with_identifier_module(Arc::new(eids::PrebidEids)) .with_deferred_js() .build(), )) @@ -1122,6 +1127,7 @@ pub fn register( IntegrationRegistration::builder(PREBID_INTEGRATION_ID) .with_proxy(integration.clone()) .with_middleware(Arc::new(PageChange::new(integration))) + .with_identifier_module(Arc::new(eids::PrebidEids)) .with_deferred_js() .build(), )) diff --git a/crates/trusted-server-adapter-axum/src/app.rs b/crates/trusted-server-adapter-axum/src/app.rs index 0f340df766..ee49bcfe72 100644 --- a/crates/trusted-server-adapter-axum/src/app.rs +++ b/crates/trusted-server-adapter-axum/src/app.rs @@ -308,6 +308,7 @@ impl AppState { if let Some(module) = self.registry.device_module() { services = services.with_device_module(module); } + services = services.with_identifier_modules(self.registry.identifier_modules()); if let Some(module) = self.registry.asset_cache_module() { services = services.with_asset_cache_module(module); } diff --git a/crates/trusted-server-adapter-cloudflare/src/app.rs b/crates/trusted-server-adapter-cloudflare/src/app.rs index 9240e078fc..59ff6030bb 100644 --- a/crates/trusted-server-adapter-cloudflare/src/app.rs +++ b/crates/trusted-server-adapter-cloudflare/src/app.rs @@ -353,6 +353,7 @@ impl AppState { if let Some(module) = self.registry.device_module() { services = services.with_device_module(module); } + services = services.with_identifier_modules(self.registry.identifier_modules()); if let Some(module) = self.registry.asset_cache_module() { services = services.with_asset_cache_module(module); } diff --git a/crates/trusted-server-adapter-fastly/src/app.rs b/crates/trusted-server-adapter-fastly/src/app.rs index 107c8a69df..a9ec7cf59e 100644 --- a/crates/trusted-server-adapter-fastly/src/app.rs +++ b/crates/trusted-server-adapter-fastly/src/app.rs @@ -53,8 +53,8 @@ //! - [`build_ec_request_state`] runs before every dispatched route (except //! batch-sync, which uses Bearer auth) //! and reproduces the legacy -//! pre-routing prelude: device signals, bot gate, `ts-eids`/`sharedid` -//! cookie capture, geo lookup, [`EcContext`] creation, and KV-graph gating. +//! pre-routing prelude: device signals, bot gate, `sharedId` cookie +//! capture, geo lookup, [`EcContext`] creation, and KV-graph gating. //! - `handle_auction` and integration proxy dispatch receive the same //! [`EcContext`], [`KvIdentityGraph`], and [`PartnerRegistry`] inputs as //! legacy; the publisher fallback generates EC IDs for browser navigations. @@ -100,7 +100,7 @@ use trusted_server_core::auction::{ use trusted_server_core::cache_policy::EdgeCacheHeader; use trusted_server_core::closed_paths::closed_path_response; use trusted_server_core::config_payload::DEFAULT_SECRET_STORE_ID; -use trusted_server_core::constants::{COOKIE_EIDS, COOKIE_SHAREDID}; +use trusted_server_core::constants::COOKIE_SHAREDID; use trusted_server_core::ec::batch_sync::handle_batch_sync; use trusted_server_core::ec::device::DeviceSignals; use trusted_server_core::ec::identify::{cors_preflight_identify, handle_identify}; @@ -532,6 +532,7 @@ fn build_per_request_services(state: &AppState, ctx: &RequestContext) -> Runtime if let Some(module) = state.registry.asset_cache_module() { services = services.with_asset_cache_module(module); } + let services = services.with_identifier_modules(state.registry.identifier_modules()); // The publisher this request resolved to, its tenant included, names // every key in a shared store and every name the browser holds. let prefixes = state.registry.prefixes( @@ -610,7 +611,6 @@ pub(crate) struct EcFinalizeState { /// in response extensions, so `edgezero_main` rebuilds the graph from /// settings when this is set. pub(crate) use_finalize_kv: bool, - pub(crate) eids_cookie: Option, pub(crate) sharedid_cookie: Option, pub(crate) is_real_browser: bool, /// Per-request services carried to the entry point so the pull-sync @@ -628,7 +628,6 @@ struct EcRequestState { ec_context: EcContext, kv_graph: Option, finalize_kv_graph: Option, - eids_cookie: Option, sharedid_cookie: Option, is_real_browser: bool, services: RuntimeServices, @@ -648,7 +647,6 @@ impl EcRequestState { EcFinalizeState { ec_context: self.ec_context, use_finalize_kv: self.finalize_kv_graph.is_some(), - eids_cookie: self.eids_cookie, sharedid_cookie: self.sharedid_cookie, is_real_browser: self.is_real_browser, services: self.services, @@ -706,8 +704,6 @@ async fn build_ec_request_state( ); } - let eids_cookie = - crate::extract_cookie_value(req, &services.public_prefix().cookie_name(COOKIE_EIDS)); let sharedid_cookie = crate::extract_cookie_value(req, COOKIE_SHAREDID); let (ec_context, setup_error) = @@ -746,7 +742,6 @@ async fn build_ec_request_state( ec_context, kv_graph, finalize_kv_graph, - eids_cookie, sharedid_cookie, is_real_browser, services: services.clone(), @@ -1034,8 +1029,6 @@ async fn run_named_route( fn run_batch_sync(state: &AppState, services: &RuntimeServices, req: Request) -> Response { let device_signals = device_signals_for(&req); let is_real_browser = device_signals.looks_like_browser; - let eids_cookie = - crate::extract_cookie_value(&req, &services.public_prefix().cookie_name(COOKIE_EIDS)); let sharedid_cookie = crate::extract_cookie_value(&req, COOKIE_SHAREDID); let result = crate::require_identity_graph(&state.settings).and_then(|kv| { @@ -1054,7 +1047,6 @@ fn run_batch_sync(state: &AppState, services: &RuntimeServices, req: Request) -> response.extensions_mut().insert(EcFinalizeState { ec_context: EcContext::default(), use_finalize_kv: false, - eids_cookie, sharedid_cookie, is_real_browser, services: services.clone(), diff --git a/crates/trusted-server-adapter-fastly/src/lib.rs b/crates/trusted-server-adapter-fastly/src/lib.rs index 4db46b648b..0637eecf2f 100644 --- a/crates/trusted-server-adapter-fastly/src/lib.rs +++ b/crates/trusted-server-adapter-fastly/src/lib.rs @@ -984,7 +984,6 @@ async fn apply_edgezero_ec_finalize( &mut ec_state.ec_context, finalize_kv_graph.as_ref(), &partner_registry, - ec_state.eids_cookie.as_deref(), ec_state.sharedid_cookie.as_deref(), response, &ec_state.services, diff --git a/crates/trusted-server-adapter-spin/src/app.rs b/crates/trusted-server-adapter-spin/src/app.rs index 0dd1d88a2f..8255dbc15f 100644 --- a/crates/trusted-server-adapter-spin/src/app.rs +++ b/crates/trusted-server-adapter-spin/src/app.rs @@ -365,6 +365,7 @@ impl AppState { if let Some(module) = self.registry.device_module() { services = services.with_device_module(module); } + services = services.with_identifier_modules(self.registry.identifier_modules()); if let Some(module) = self.registry.asset_cache_module() { services = services.with_asset_cache_module(module); } diff --git a/crates/trusted-server-core/src/auction/endpoints.rs b/crates/trusted-server-core/src/auction/endpoints.rs index 2a8c7392b0..023f8cd287 100644 --- a/crates/trusted-server-core/src/auction/endpoints.rs +++ b/crates/trusted-server-core/src/auction/endpoints.rs @@ -10,14 +10,11 @@ use serde_json::Value as JsonValue; use crate::auction::formats::AdRequest; use crate::auction::orchestrator::OrchestrationResult; use crate::consent::{consent_allows_server_side_auction, gate_eids_by_permissions}; -use crate::constants::COOKIE_EIDS; -use crate::cookies::extract_cookie_value; use crate::ec::EcContext; use crate::ec::EcKvSnapshot; use crate::ec::eids::{resolve_partner_ids, to_eids}; use crate::ec::kv::KvIdentityGraph; use crate::ec::kv_types::MAX_UID_LENGTH; -use crate::ec::prebid_eids::parse_prebid_eids_cookie; use crate::ec::registry::PartnerRegistry; use crate::error::TrustedServerError; use crate::openrtb::{Eid, Uid}; @@ -279,8 +276,8 @@ pub async fn handle_auction( } // Parse client-provided EIDs from the current request body. When the - // current request does not include them, fall back to the persisted - // `ts-eids` cookie so later requests can still forward the browser's + // current request does not include them, fall back to what the identifier + // modules learned, so later requests can still forward the browser's // full OpenRTB-style EID structure. // // Gate this on the same identity condition as the EC ID @@ -293,20 +290,13 @@ pub async fn handle_auction( // `/_ts/page-bids` paths, which also resolve client EIDs only when // `ec_id.is_some()`. let client_eids = if ec_id.is_some() { - resolve_client_auction_eids( - body.eids.as_ref(), - extract_cookie_value( - &http_req, - &services.public_prefix().cookie_name(COOKIE_EIDS), - ) - .as_deref(), - ) + resolve_client_auction_eids(body.eids.as_ref(), ec_context.module_eids()) } else { None }; // Carry the full request-local EID set to response finalization so KV // ingestion uses what this request actually sent, not just whatever fits - // in the size-capped `ts-eids` cookie (see `ec::finalize::ec_finalize_response`). + // in a module's size-capped cookie (see `ec::finalize::ec_finalize_response`). if let Some(eids) = &client_eids { ec_context.set_client_eids(eids.clone()); } @@ -324,10 +314,10 @@ pub async fn handle_auction( .map_or(EcKvSnapshot::NotRead, |kv_key| graph.load_snapshot(&kv_key)), _ => EcKvSnapshot::NotRead, }; - // Hand the loaded row to the request context so response finalization — - // which runs on an EC context the adapter owns, after this handler returns - // — ingests `ts-eids`/`sharedId` updates from this read instead of paying - // for a second lookup. + // Hand the loaded row to the request context so response finalization, + // which runs on an EC context the adapter owns after this handler + // returns, ingests the request's identifier updates from this read + // instead of paying for a second lookup. if !matches!(auction_kv_snapshot, EcKvSnapshot::NotRead) { ec_context.set_kv_snapshot(auction_kv_snapshot.clone()); } @@ -504,23 +494,14 @@ pub(crate) fn resolve_auction_eids( Some(to_eids(&resolved)) } +/// The EIDs the request body carries, or what the identifier modules learned +/// when the body carries none. pub(crate) fn resolve_client_auction_eids( raw: Option<&JsonValue>, - cookie_value: Option<&str>, + module_eids: &[Eid], ) -> Option> { - parse_client_auction_eids(raw).or_else(|| parse_cookie_auction_eids(cookie_value)) -} - -fn parse_cookie_auction_eids(cookie_value: Option<&str>) -> Option> { - let cookie_value = cookie_value?; - match parse_prebid_eids_cookie(cookie_value) { - Ok(eids) if eids.is_empty() => None, - Ok(eids) => Some(eids), - Err(_) => { - log::trace!("Auction EIDs: failed to parse ts-eids cookie; dropping"); - None - } - } + parse_client_auction_eids(raw) + .or_else(|| (!module_eids.is_empty()).then(|| module_eids.to_vec())) } fn parse_client_auction_eids(raw: Option<&JsonValue>) -> Option> { @@ -680,8 +661,6 @@ mod tests { PlatformResponse, }; use crate::test_support::tests::{crate_test_settings_str, create_test_settings}; - use base64::Engine as _; - use base64::engine::general_purpose::STANDARD as BASE64; use serde_json::json; use std::sync::{Arc, Mutex}; use std::time::Duration; @@ -768,7 +747,7 @@ mod tests { async fn auction_endpoint_snapshot_is_reused_by_response_finalization() { // `/auction` loads the identity-graph row to resolve server-side EIDs. // Finalization runs afterwards on the same EC context and ingests - // `ts-eids`/`sharedId` updates. Both must be served by a single billable + // the request's identifier updates. Both must be served by a single billable // read: before the snapshot was shared, finalization saw `NotRead` and // paid for a second lookup. let settings = create_test_settings(); @@ -840,7 +819,6 @@ mod tests { &mut ec_context, Some(&graph), ®istry, - None, Some("shared-cookie-id"), &mut response, &crate::platform::test_support::noop_services(), @@ -942,13 +920,12 @@ mod tests { } #[tokio::test] - async fn auction_body_eids_reach_kv_even_when_the_ts_eids_cookie_is_absent() { - // Regression test for #1184: `/auction` sends every EID it has in the - // request body, but response finalization used to ingest identity - // graph updates only from the `ts-eids` cookie — which the browser - // caps in size and may not have sent at all. `handle_auction` must - // hand its parsed body EIDs to `ec_context` so finalization ingests - // them regardless of what the cookie carried. + async fn auction_body_eids_reach_kv_even_when_no_module_learned_any() { + // `/auction` sends every EID it has in the request body, and a + // module's cookie is capped in size and may not have been sent at + // all. `handle_auction` must hand its parsed body EIDs to + // `ec_context` so finalization ingests them regardless of what the + // modules learned. let settings = create_test_settings(); let mut orchestrator = AuctionOrchestrator::new(AuctionConfig { enabled: true, @@ -1026,7 +1003,6 @@ mod tests { &mut ec_context, Some(&graph), ®istry, - None, // No `ts-eids` cookie on this request at all. None, &mut response, &noop_services(), @@ -1040,7 +1016,7 @@ mod tests { assert_eq!( stored.ids.get("id5-sync.com").map(|id| id.uid.as_str()), Some("ID5_from_body"), - "the body's EID must be ingested into KV without a ts-eids cookie" + "the body's EID must be ingested into KV when no module learned any" ); } @@ -1546,15 +1522,16 @@ mod tests { }, ); - // Persistent EIDs supplied in both the request body and the ts-eids cookie. - let cookie_payload = json!([ - { - "source": "sharedid.org", - "uids": [{ "id": "cookie_uid", "atype": 3 }] - } - ]); - let encoded_cookie = BASE64 - .encode(serde_json::to_vec(&cookie_payload).expect("should serialize cookie payload")); + // Persistent EIDs supplied both in the request body and by an + // identifier module. + ec_context.set_module_eids_for_test(vec![Eid { + source: "sharedid.org".to_owned(), + uids: vec![Uid { + id: "module_uid".to_owned(), + atype: Some(3), + ext: None, + }], + }]); let body = json!({ "adUnits": [ { @@ -1572,7 +1549,6 @@ mod tests { let req = Request::builder() .method("POST") .uri("https://test-publisher.com/auction") - .header("cookie", format!("ts-{COOKIE_EIDS}={encoded_cookie}")) .body(EdgeBody::from( serde_json::to_vec(&body).expect("should serialize body"), )) @@ -1669,56 +1645,66 @@ mod tests { ); } - #[test] - fn resolve_client_auction_eids_falls_back_to_ts_eids_cookie() { - let cookie_payload = json!([ - { - "source": "sharedid.org", - "uids": [ - { "id": "shared_cookie", "atype": 3 }, - { "id": "shared_cookie_2", "ext": { "provider": "example" } } - ] - } - ]); - let encoded = BASE64 - .encode(serde_json::to_vec(&cookie_payload).expect("should serialize cookie payload")); + fn module_eids_for_tests() -> Vec { + vec![Eid { + source: "sharedid.org".to_owned(), + uids: vec![ + Uid { + id: "shared_module".to_owned(), + atype: Some(3), + ext: None, + }, + Uid { + id: "shared_module_2".to_owned(), + atype: None, + ext: Some(json!({ "provider": "example" })), + }, + ], + }] + } - let resolved = resolve_client_auction_eids(None, Some(&encoded)) - .expect("should fall back to structured ts-eids cookie"); + #[test] + fn resolve_client_auction_eids_falls_back_to_module_eids() { + let resolved = resolve_client_auction_eids(None, &module_eids_for_tests()) + .expect("should fall back to what the modules learned"); - assert_eq!(resolved.len(), 1, "should preserve cookie source entry"); + assert_eq!( + resolved.len(), + 1, + "should preserve the module's source entry" + ); assert_eq!(resolved[0].source, "sharedid.org"); assert_eq!( resolved[0].uids.len(), 2, - "should preserve multiple cookie UIDs" + "should preserve multiple module UIDs" ); - assert_eq!(resolved[0].uids[0].id, "shared_cookie"); + assert_eq!(resolved[0].uids[0].id, "shared_module"); assert_eq!( resolved[0].uids[1].ext, Some(json!({ "provider": "example" })), - "should preserve UID ext from cookie fallback" + "should preserve UID ext from the module fallback" + ); + } + + #[test] + fn resolve_client_auction_eids_is_none_when_nothing_was_learned() { + assert!( + resolve_client_auction_eids(None, &[]).is_none(), + "no body and nothing learned should resolve to no EIDs" ); } #[test] - fn resolve_client_auction_eids_prefers_request_body_over_cookie() { + fn resolve_client_auction_eids_prefers_request_body_over_module_eids() { let raw = json!([ { "source": "id5-sync.com", "uids": [{ "id": "body_uid", "atype": 1 }] } ]); - let cookie_payload = json!([ - { - "source": "sharedid.org", - "uids": [{ "id": "cookie_uid", "atype": 3 }] - } - ]); - let encoded = BASE64 - .encode(serde_json::to_vec(&cookie_payload).expect("should serialize cookie payload")); - let resolved = resolve_client_auction_eids(Some(&raw), Some(&encoded)) + let resolved = resolve_client_auction_eids(Some(&raw), &module_eids_for_tests()) .expect("should prefer request body EIDs"); assert_eq!(resolved.len(), 1, "should use request body when present"); diff --git a/crates/trusted-server-core/src/auction/types.rs b/crates/trusted-server-core/src/auction/types.rs index f8d83d7f9d..27943f1f13 100644 --- a/crates/trusted-server-core/src/auction/types.rs +++ b/crates/trusted-server-core/src/auction/types.rs @@ -96,7 +96,8 @@ pub struct UserInfo { /// cookies/headers, not from stored data. #[serde(skip)] pub consent: Option, - /// Extended User IDs parsed from the [`crate::constants::COOKIE_EIDS`] cookie. + /// Extended User IDs the request carried, from its body or from an + /// identifier module. /// /// Raw (un-gated) values from the browser; consent gating via /// [`crate::consent::gate_eids_by_consent`] is applied centrally in the diff --git a/crates/trusted-server-core/src/constants.rs b/crates/trusted-server-core/src/constants.rs index 9aa5a5dd9b..f32bbd411d 100644 --- a/crates/trusted-server-core/src/constants.rs +++ b/crates/trusted-server-core/src/constants.rs @@ -12,9 +12,6 @@ pub const COOKIE_EC_RESOLVED: &str = "ecr"; /// Short-lived signed proof that the current EC row has every pull-partner UID. pub const COOKIE_EC_PULL_COMPLETE: &str = "ec-pull-complete"; -/// Cookie written by the Trusted Server JS SDK containing a standard-base64-encoded -/// JSON array of Extended User IDs (`[{ source, uids }]`) from identity providers. -pub const COOKIE_EIDS: &str = "eids"; /// The cookie the tester routes set and clear. pub const COOKIE_TESTER: &str = "tester"; pub const COOKIE_SHAREDID: &str = "sharedId"; diff --git a/crates/trusted-server-core/src/cookies/template_cache_policy.rs b/crates/trusted-server-core/src/cookies/template_cache_policy.rs index 71d9318a50..072ee47048 100644 --- a/crates/trusted-server-core/src/cookies/template_cache_policy.rs +++ b/crates/trusted-server-core/src/cookies/template_cache_policy.rs @@ -2,19 +2,16 @@ use std::collections::{HashMap, HashSet}; -use crate::constants::{COOKIE_EC, COOKIE_EIDS, COOKIE_SHAREDID}; +use crate::constants::COOKIE_SHAREDID; use crate::platform::TemplateCookieValue; use crate::prefix::PublicPrefix; -/// The identity cookies a template key must never be built from, under -/// `public`, the request's public prefix, being the Edge Cookie, the EIDs -/// cookie and `sharedId`. -pub(crate) fn identity_cookies(public: &PublicPrefix) -> [String; 3] { - [ - public.cookie_name(COOKIE_EC), - public.cookie_name(COOKIE_EIDS), - COOKIE_SHAREDID.to_owned(), - ] +/// Whether `name` is an identity cookie a template key must never be built +/// from, under `public`, the request's public prefix. Every cookie under the +/// prefix is one reader's own state, the Edge Cookie and a module's +/// identifier cookie among them, and `sharedId` is one too. +pub(crate) fn is_identity_cookie(public: &PublicPrefix, name: &str) -> bool { + public.owns_cookie_name(name.as_bytes()) || name == COOKIE_SHAREDID } /// Whether the request can share a template and its explicit cookie dimensions. @@ -31,12 +28,12 @@ pub(crate) enum TemplateCookieDecision { /// Returns an error naming the field and cookie when a name is invalid or repeated, /// or when the key list names an identity cookie under core's own prefix. A /// deployment's own prefix is checked on each request, see -/// [`identity_cookies`]. +/// [`is_identity_cookie`]. pub(crate) fn validate_cookie_names( key_names: &[String], bypass_names: &[String], ) -> Result<(), String> { - let identity = identity_cookies(&PublicPrefix::default()); + let public = PublicPrefix::default(); let mut seen = HashSet::new(); for (field, names) in [ ("template_cache_key_cookies", key_names), @@ -46,7 +43,7 @@ pub(crate) fn validate_cookie_names( if !is_cookie_name(name.as_bytes()) { return Err(format!("{field} contains invalid cookie name `{name}`")); } - if field == "template_cache_key_cookies" && identity.contains(name) { + if field == "template_cache_key_cookies" && is_identity_cookie(&public, name) { return Err(format!( "{field} must not key on Trusted Server identity cookie `{name}`" )); @@ -245,7 +242,13 @@ mod tests { #[test] fn template_cookie_policy_rejects_identity_keys_but_allows_bypass() { - for name in identity_cookies(&PublicPrefix::default()) { + let public = PublicPrefix::default(); + for name in [ + public.cookie_name(crate::constants::COOKIE_EC), + public.cookie_name(crate::constants::COOKIE_EC_RESOLVED), + public.cookie_name("a-module-cookie"), + COOKIE_SHAREDID.to_owned(), + ] { let name = name.as_str(); let error = validate_cookie_names(&names(&[name]), &[]) .expect_err("should reject identity cookie keys"); diff --git a/crates/trusted-server-core/src/ec/finalize.rs b/crates/trusted-server-core/src/ec/finalize.rs index 46bf43290e..e160c374ee 100644 --- a/crates/trusted-server-core/src/ec/finalize.rs +++ b/crates/trusted-server-core/src/ec/finalize.rs @@ -13,21 +13,22 @@ use crate::settings::Settings; use super::EcContext; use super::cookies::{expire_ec_cookie, expire_ec_resolved_marker, set_ec_cookie}; +use super::identifiers::collect_eid_updates; use super::kv::{ CreateIfAbsentOutcome, EidCookieSyncOutcome, KvIdentityGraph, PartnerIdUpdate, apply_partner_id_updates, }; use super::kv_types::KvEntry; use super::module::{apply_module_response_headers, module_kv_key}; -use super::prebid_eids::collect_eid_updates; use super::pull_sync_marker::{expire_marker, reconcile_marker}; use super::registry::PartnerRegistry; use super::{EcKvSnapshot, EidSyncSource, current_timestamp, log_id}; /// Finalizes EC response behavior for all routes. /// -/// Applies the resolved permission state, cookie reconciliation, Prebid EID -/// ingestion, and cookie writes for new EC generation. +/// Applies the resolved permission state, cookie reconciliation, the +/// ingestion of the identifiers the request carried, and cookie writes for +/// new EC generation. /// /// When the request carries an explicit withdrawal signal (a storage opt-out or /// a TCF record refusing storage) and the client presented a cookie, the browser @@ -37,21 +38,19 @@ use super::{EcKvSnapshot, EidSyncSource, current_timestamp, log_id}; /// leaves an already-issued cookie intact. There is no separate consent KV /// store to clean up. /// -/// `eids_cookie` should be the raw value of the EIDs cookie extracted -/// from the request *before* routing consumes it. +/// The identifiers the request carried come from `ec_context`, which holds +/// what the identifier modules learned when the state was read and what the +/// route captured from the request body. `sharedid_cookie` should be the raw +/// value of the `sharedId` cookie extracted from the request *before* routing +/// consumes it. /// /// `services` are the request's runtime services, handed to the selected Edge /// Cookie module when an orphaned identifier is replaced. -#[allow( - clippy::too_many_arguments, - reason = "orphan recovery asks the selected module for a replacement identifier, which needs the request's services on top of the finalize inputs" -)] pub async fn ec_finalize_response( settings: &Settings, ec_context: &mut EcContext, kv: Option<&KvIdentityGraph>, registry: &PartnerRegistry, - eids_cookie: Option<&str>, sharedid_cookie: Option<&str>, response: &mut Response, services: &crate::platform::RuntimeServices, @@ -119,7 +118,7 @@ pub async fn ec_finalize_response( let collected = source .map(|_| { collect_eid_updates( - eids_cookie, + ec_context.module_eids(), sharedid_cookie, ec_context.client_eids(), registry, @@ -181,7 +180,7 @@ pub async fn ec_finalize_response( let updates = gate_eid_updates_by_permissions( collect_eid_updates( - eids_cookie, + ec_context.module_eids(), sharedid_cookie, ec_context.client_eids(), registry, @@ -295,8 +294,8 @@ fn reconcile_pull_sync_marker( /// applies to the outbound `/auction` bid request, answered here by /// [`EcContext::ec_sharing_allowed`]. Without this, a reader who withheld the /// personalised-ad permission would have partner UIDs stripped from the bid -/// request but still written to the identity graph from the `ts-eids` and -/// `sharedId` cookies or the `/auction` request body. +/// request but still written to the identity graph from what the identifier +/// modules learned, the `sharedId` cookie or the `/auction` request body. fn gate_eid_updates_by_permissions( updates: Vec, ec_context: &EcContext, @@ -617,8 +616,6 @@ mod tests { use crate::platform::test_support::noop_services; use http::HeaderValue; - use base64::Engine as _; - use super::*; use crate::consent::jurisdiction::Jurisdiction; use crate::consent::types::{ConsentContext, ConsentSource}; @@ -878,7 +875,6 @@ mod tests { Some(&kv), ®istry, None, - None, &mut response, &noop_services(), ) @@ -936,7 +932,6 @@ mod tests { Some(&kv), ®istry, None, - None, &mut response, &noop_services(), ) @@ -993,7 +988,6 @@ mod tests { Some(&kv), ®istry, None, - None, &mut response, &noop_services(), ) @@ -1046,7 +1040,6 @@ mod tests { None, &test_registry, None, - None, &mut response, &noop_services(), ) @@ -1124,7 +1117,6 @@ mod tests { None, &test_registry, None, - None, &mut response, &noop_services(), ) @@ -1162,7 +1154,6 @@ mod tests { None, &test_registry, None, - None, &mut response, &noop_services(), ) @@ -1199,7 +1190,6 @@ mod tests { None, &test_registry, None, - None, &mut response, &noop_services(), ) @@ -1236,7 +1226,6 @@ mod tests { None, &test_registry, None, - None, &mut response, &noop_services(), ) @@ -1280,7 +1269,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -1345,7 +1333,6 @@ mod tests { Some(&graph), ®istry, None, - None, &mut response, &noop_services(), ) @@ -1402,7 +1389,6 @@ mod tests { Some(&graph), ®istry, None, - None, &mut response, &noop_services(), ) @@ -1447,7 +1433,6 @@ mod tests { &mut ec_context, Some(&graph), ®istry, - None, Some("shared-cookie-id"), &mut response, &noop_services(), @@ -1493,7 +1478,6 @@ mod tests { &mut ec_context, Some(&graph), ®istry, - None, Some(cookie_id), &mut response, &noop_services(), @@ -1544,7 +1528,6 @@ mod tests { &mut ec_context, Some(&graph), ®istry, - None, Some("generated-cookie-id"), &mut response, &noop_services(), @@ -1691,7 +1674,6 @@ mod tests { &mut ec_context, Some(&graph), ®istry, - None, Some("shared-cookie-id"), &mut response, &noop_services(), @@ -1712,11 +1694,11 @@ mod tests { #[tokio::test] async fn finalize_persists_every_configured_partner_from_client_eids_over_a_trimmed_cookie() { - // Regression test for #1184: the `ts-eids` cookie only carries what - // the browser could fit under its size cap, but `/auction` also - // hands finalization the full EID set from the request body via - // `EcContext::set_client_eids`. That full set must land in KV even - // when the cookie alone would have dropped a configured partner. + // A module's cookie only carries what the browser could fit under + // its size cap, but `/auction` also hands finalization the full EID + // set from the request body via `EcContext::set_client_eids`. That + // full set must land in KV even when what the module learned alone + // would have dropped a configured partner. let settings = create_test_settings(); let ec_id = sample_ec_id("cleid1"); let graph = KvIdentityGraph::in_memory("test_store"); @@ -1737,14 +1719,16 @@ mod tests { false, ); ec_context.set_eid_sync_source(EidSyncSource::Auction); - // Only `id5-sync.com` "fit" in the (simulated) trimmed cookie; - // `liveramp.com` was dropped by the browser's size cap. - let eids_cookie = base64::engine::general_purpose::STANDARD.encode( - serde_json::to_vec(&serde_json::json!([ - {"source": "id5-sync.com", "uids": [{"id": "ID5_from_cookie", "atype": 1}]} - ])) - .expect("should serialize test cookie payload"), - ); + // Only `id5-sync.com` "fit" in the (simulated) trimmed cookie the + // module read; `liveramp.com` was dropped by the browser's size cap. + ec_context.set_module_eids_for_test(vec![Eid { + source: "id5-sync.com".to_owned(), + uids: vec![Uid { + id: "ID5_from_cookie".to_owned(), + atype: Some(1), + ext: None, + }], + }]); ec_context.set_client_eids(vec![ Eid { source: "id5-sync.com".to_owned(), @@ -1772,7 +1756,6 @@ mod tests { &mut ec_context, Some(&graph), ®istry, - Some(&eids_cookie), None, &mut response, &noop_services(), @@ -1846,7 +1829,6 @@ mod tests { Some(&graph), ®istry, None, - None, &mut response, &noop_services(), ) @@ -1887,7 +1869,6 @@ mod tests { &mut ec_context, Some(&graph), ®istry, - None, Some("shared-cookie-id"), &mut response, &noop_services(), @@ -1934,7 +1915,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -1983,7 +1963,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -2042,7 +2021,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -2196,7 +2174,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -2245,7 +2222,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -2306,7 +2282,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -2339,7 +2314,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -2364,7 +2338,6 @@ mod tests { None, &test_registry, None, - None, &mut response, &noop_services(), ) @@ -2408,7 +2381,6 @@ mod tests { None, &test_registry, None, - None, &mut response, &noop_services(), ) @@ -2487,7 +2459,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -2538,7 +2509,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -2584,7 +2554,6 @@ mod tests { &mut ec_context, Some(&graph), ®istry, - None, Some("shared-cookie-id"), &mut response, &noop_services(), @@ -2691,7 +2660,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -2826,7 +2794,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -2881,7 +2848,6 @@ mod tests { None, &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -2918,7 +2884,6 @@ mod tests { None, &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -2991,7 +2956,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -3020,7 +2984,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -3048,7 +3011,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -3078,7 +3040,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -3129,7 +3090,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -3191,7 +3151,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -3229,7 +3188,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut repeated_response, &noop_services(), ) @@ -3280,7 +3238,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -3349,7 +3306,6 @@ mod tests { None, ®istry, None, - None, &mut response, &noop_services(), ) @@ -3387,7 +3343,6 @@ mod tests { None, &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -3421,7 +3376,6 @@ mod tests { None, &PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) diff --git a/crates/trusted-server-core/src/ec/identifiers.rs b/crates/trusted-server-core/src/ec/identifiers.rs new file mode 100644 index 0000000000..945f56b0ea --- /dev/null +++ b/crates/trusted-server-core/src/ec/identifiers.rs @@ -0,0 +1,622 @@ +//! Identifiers a module learns for a request, and their sync into the +//! identity graph. +//! +//! An [`IdentifierModule`] hands core the identifiers it learned for the +//! request, such as the ones a browser-side script persisted in a cookie the +//! module owns. Core asks every running module once, when it reads the Edge +//! Cookie state, and carries the answers on the request's +//! [`EcContext`](super::EcContext) to response finalization, which offers +//! them to the identity graph, and to the auction, which forwards them when +//! the request body carries none. +//! +//! Core names no cookie here. A module reads its own, through the request's +//! evidence and the public prefix the services carry, and returns what it +//! found. A module is offered through +//! [`IntegrationRegistrationBuilder::with_identifier_module`](crate::integrations::IntegrationRegistrationBuilder::with_identifier_module). + +use std::sync::Arc; + +use crate::module_context::{ModuleCall, ModuleContext, Withheld}; +use crate::openrtb::{Eid, Uid}; +use crate::permissions::PermissionSet; + +use super::kv::PartnerIdUpdate; +use super::kv_types::MAX_UID_LENGTH; +use super::registry::PartnerRegistry; + +/// Learns identifiers for a request. +/// +/// Each call is handed a [`ModuleCall`] carrying the request's evidence, +/// settings, services, consent and resolved permissions, as every module is, +/// and an implementation hands its own function to [`ModuleCall::inject`] +/// naming what it reads. A module reads what the request carries and retains +/// nothing. +pub trait IdentifierModule: Send + Sync + core::fmt::Debug { + /// The module's name, for the call and the logs. + fn id(&self) -> &'static str; + + /// The permissions this module declares, which decide whether a gated + /// value is passed to it. None by default. + fn required_permissions(&self) -> PermissionSet { + PermissionSet::none() + } + + /// The identifiers the module learned for this request, in the order it + /// offers them, so that where two carry the same partner the later one is + /// the candidate. Empty when the request carries none. + /// + /// # Errors + /// + /// [`Withheld`] when the module names a value the call does not carry. + fn identifiers(&self, call: ModuleCall<'_>) -> Result, Withheld>; +} + +/// Asks each of `modules` for the identifiers it learned, in order. +/// +/// A module whose call withholds a value it names contributes nothing and is +/// logged, so one module's fault does not lose another's identifiers. +pub(crate) fn learn( + context: &ModuleContext<'_>, + modules: &[Arc], +) -> Vec { + let mut learned = Vec::new(); + for module in modules { + match module.identifiers(context.call(module.id(), module.required_permissions())) { + Ok(eids) => learned.extend(eids), + Err(withheld) => log::warn!( + "identifier module `{}` contributed nothing: {withheld}", + module.id() + ), + } + } + learned +} + +/// Collects validated request-local partner updates without performing KV I/O. +/// +/// `module_eids` are what the identifier modules learned, applied first. +/// `body_eids` are the current request's own EIDs (e.g. an `/auction` JSON +/// body), when the route captured any, applied after so they win on +/// conflicts: a cookie a module read is size-capped by the browser and may +/// be missing partners the request body still carries in full. The +/// `sharedId` cookie is applied last, unaffected by `body_eids`. +pub(crate) fn collect_eid_updates( + module_eids: &[Eid], + sharedid_cookie: Option<&str>, + body_eids: Option<&[Eid]>, + registry: &PartnerRegistry, +) -> Vec { + if registry.is_empty() { + return Vec::new(); + } + + let mut updates = collect_eid_updates_from_eids(module_eids, registry); + if let Some(eids) = body_eids { + updates.extend(collect_eid_updates_from_eids(eids, registry)); + } + if let Some(cookie) = sharedid_cookie + && let Some(update) = collect_sharedid_update(cookie, registry) + { + updates.push(update); + } + dedupe_partner_updates(updates) +} + +fn collect_eid_updates_from_eids(eids: &[Eid], registry: &PartnerRegistry) -> Vec { + let mut updates = Vec::new(); + for eid in eids { + // An EID's source is what the reader sent, so no log line carries + // it. A line names the partner's configured domain instead. + let Some(partner) = registry.find_by_source_domain(&eid.source) else { + log::debug!("EIDs: skipped an EID whose source matches no partner"); + continue; + }; + + // KV stores one UID per partner. Sync the first valid UID under each + // source, skipping malformed candidates instead of dropping the whole + // source. + let Some(uid) = first_valid_uid(&eid.uids) else { + log::debug!( + "EIDs: no valid uid for source_domain '{}'", + partner.source_domain + ); + continue; + }; + + updates.push(PartnerIdUpdate::new(&partner.source_domain, &uid.id)); + } + + updates +} + +fn dedupe_partner_updates(updates: Vec) -> Vec { + let mut latest = std::collections::BTreeMap::new(); + for update in updates { + latest.insert(update.partner_id, update.uid); + } + + latest + .into_iter() + .map(|(partner_id, uid)| PartnerIdUpdate::new(partner_id, uid)) + .collect() +} + +fn first_valid_uid(uids: &[Uid]) -> Option<&Uid> { + uids.iter().find(|uid| is_valid_eid_uid(&uid.id)) +} + +fn is_valid_eid_uid(uid: &str) -> bool { + !uid.trim().is_empty() && !eid_id_exceeds_size_limit(uid) +} + +/// `SharedID` EID source domain used for partner registry lookup. +const SHAREDID_SOURCE_DOMAIN: &str = "sharedid.org"; + +fn collect_sharedid_update( + cookie_value: &str, + registry: &PartnerRegistry, +) -> Option { + let cookie_value = cookie_value.trim(); + if cookie_value.is_empty() { + return None; + } + + if sharedid_cookie_exceeds_size_limit(cookie_value) { + log::debug!( + "SharedID: cookie exceeds MAX_UID_LENGTH ({} bytes)", + cookie_value.len() + ); + return None; + } + + let Some(partner) = registry.find_by_source_domain(SHAREDID_SOURCE_DOMAIN) else { + log::debug!("SharedID: no partner configured for source '{SHAREDID_SOURCE_DOMAIN}'"); + return None; + }; + + Some(PartnerIdUpdate::new(&partner.source_domain, cookie_value)) +} + +fn eid_id_exceeds_size_limit(uid: &str) -> bool { + uid.len() > MAX_UID_LENGTH +} + +fn sharedid_cookie_exceeds_size_limit(cookie_value: &str) -> bool { + cookie_value.len() > MAX_UID_LENGTH +} + +#[cfg(test)] +mod tests { + use super::*; + + use crate::ec::registry::PartnerRegistry; + use crate::evidence::{OwnedRequestInfo, RequestInfo}; + use crate::module_context::test_support::request; + use crate::redacted::Redacted; + use crate::settings::EcPartner; + + fn make_test_partner(source_domain: &str) -> EcPartner { + EcPartner { + name: format!("Partner {source_domain}"), + source_domain: source_domain.to_owned(), + openrtb_atype: EcPartner::default_openrtb_atype(), + bidstream_enabled: true, + api_token: Some(Redacted::new(format!( + "token-{source_domain}-32-bytes-minimum-value" + ))), + batch_rate_limit: EcPartner::default_batch_rate_limit(), + pull_sync_enabled: false, + pull_sync_url: None, + pull_sync_allowed_domains: vec![], + pull_sync_ttl_sec: EcPartner::default_pull_sync_ttl_sec(), + pull_sync_rate_limit: EcPartner::default_pull_sync_rate_limit(), + ts_pull_token: None, + } + } + + fn make_registry(source_domains: &[&str]) -> PartnerRegistry { + let partners: Vec<_> = source_domains + .iter() + .map(|source_domain| make_test_partner(source_domain)) + .collect(); + PartnerRegistry::from_config(&partners).expect("should build partner registry") + } + + fn eid(source: &str, ids: &[&str]) -> Eid { + Eid { + source: source.to_owned(), + uids: ids + .iter() + .map(|id| Uid { + id: (*id).to_owned(), + atype: Some(1), + ext: None, + }) + .collect(), + } + } + + /// A module that answers the identifiers it was built with, under a + /// fixed name as every module has. + #[derive(Debug)] + struct Fixed(Vec); + + impl IdentifierModule for Fixed { + fn id(&self) -> &'static str { + "testing.fixed" + } + + fn identifiers(&self, _call: ModuleCall<'_>) -> Result, Withheld> { + Ok(self.0.clone()) + } + } + + /// A module that reads one cookie from the request's evidence, the way a + /// module owning a cookie does. + #[derive(Debug)] + struct FromCookie; + + impl FromCookie { + fn read(&self, evidence: &dyn RequestInfo) -> Vec { + evidence + .cookie("fixture-id") + .map(|id| vec![eid("fixture.example", &[id])]) + .unwrap_or_default() + } + } + + impl IdentifierModule for FromCookie { + fn id(&self) -> &'static str { + "testing.from-cookie" + } + + fn identifiers(&self, call: ModuleCall<'_>) -> Result, Withheld> { + call.inject(self, Self::read) + } + } + + #[test] + fn learn_asks_each_module_in_order_and_keeps_their_order() { + let modules: Vec> = vec![ + Arc::new(Fixed(vec![eid("id5-sync.com", &["ID5_abc"])])), + Arc::new(Fixed(vec![eid("liveramp.com", &["LR_xyz"])])), + ]; + + let learned = learn(ModuleContext::empty(), &modules); + + assert_eq!( + learned + .iter() + .map(|eid| eid.source.as_str()) + .collect::>(), + vec!["id5-sync.com", "liveramp.com"], + "should keep the modules' order" + ); + } + + #[test] + fn learn_passes_the_evidence_a_module_names() { + let mut headers = http::HeaderMap::new(); + headers.insert( + "cookie", + "other=1; fixture-id=from-cookie" + .parse() + .expect("should build a cookie header"), + ); + let evidence = OwnedRequestInfo::new(String::new(), headers); + let context = ModuleContext::new(request("/")).with_evidence(&evidence); + let modules: Vec> = vec![Arc::new(FromCookie)]; + + let learned = learn(&context, &modules); + + assert_eq!(learned.len(), 1, "should read the cookie the module names"); + assert_eq!(learned[0].uids[0].id, "from-cookie"); + } + + #[test] + fn learn_skips_a_module_whose_call_withholds_what_it_names() { + let modules: Vec> = vec![ + Arc::new(FromCookie), + Arc::new(Fixed(vec![eid("id5-sync.com", &["ID5_abc"])])), + ]; + + let learned = learn(ModuleContext::empty(), &modules); + + assert_eq!( + learned.len(), + 1, + "a module handed no evidence should contribute nothing, and the next still answers" + ); + assert_eq!(learned[0].source, "id5-sync.com"); + } + + #[test] + fn sharedid_cookie_rejects_values_larger_than_uid_limit() { + let oversized = "x".repeat(MAX_UID_LENGTH + 1); + let exact_limit = "x".repeat(MAX_UID_LENGTH); + + assert!( + sharedid_cookie_exceeds_size_limit(&oversized), + "should reject sharedId values larger than MAX_UID_LENGTH" + ); + assert!( + !sharedid_cookie_exceeds_size_limit(&exact_limit), + "should allow sharedId values exactly at MAX_UID_LENGTH" + ); + } + + #[test] + fn eid_uid_rejects_values_larger_than_uid_limit() { + let oversized = "x".repeat(MAX_UID_LENGTH + 1); + let exact_limit = "x".repeat(MAX_UID_LENGTH); + + assert!( + eid_id_exceeds_size_limit(&oversized), + "should reject EID values larger than MAX_UID_LENGTH" + ); + assert!( + !eid_id_exceeds_size_limit(&exact_limit), + "should allow EID values exactly at MAX_UID_LENGTH" + ); + } + + #[test] + fn first_valid_uid_skips_oversized_and_uses_later_valid_uid() { + let oversized = "x".repeat(MAX_UID_LENGTH + 1); + let uids = vec![ + Uid { + id: oversized, + atype: Some(1), + ext: None, + }, + Uid { + id: "valid-uid".to_owned(), + atype: Some(1), + ext: None, + }, + ]; + + let uid = first_valid_uid(&uids).expect("should find later valid UID"); + assert_eq!(uid.id, "valid-uid", "should skip oversized first UID"); + } + + #[test] + fn first_valid_uid_rejects_whitespace_only_ids() { + let uids = vec![Uid { + id: " ".to_owned(), + atype: Some(1), + ext: None, + }]; + + assert!( + first_valid_uid(&uids).is_none(), + "should reject whitespace UID" + ); + } + + #[test] + fn collect_eid_updates_collects_multiple_partner_matches() { + let registry = make_registry(&["id5-sync.com", "liveramp.com"]); + let module_eids = vec![ + eid("id5-sync.com", &["ID5_abc"]), + eid("liveramp.com", &["LR_xyz"]), + ]; + + let updates = collect_eid_updates(&module_eids, None, None, ®istry); + + assert_eq!(updates.len(), 2, "should collect both partner matches"); + assert_eq!(updates[0], PartnerIdUpdate::new("id5-sync.com", "ID5_abc")); + assert_eq!(updates[1], PartnerIdUpdate::new("liveramp.com", "LR_xyz")); + } + + #[test] + fn collect_eid_updates_skips_unknown_sources() { + let registry = make_registry(&["id5-sync.com"]); + let module_eids = vec![eid("unknown.example", &["unknown"])]; + + let updates = collect_eid_updates(&module_eids, None, None, ®istry); + + assert!( + updates.is_empty(), + "should skip EIDs without configured source-domain partners" + ); + } + + /// Each EID skipped is logged without the source as the reader sent it, + /// and an EID a partner owns is named by the partner's configured domain. + #[test] + fn no_log_line_carries_the_source_a_reader_sent() { + let registry = make_registry(&["id5-sync.com"]); + let module_eids = vec![ + eid("reader-sent.example", &["unmatched"]), + eid("ID5-Sync.COM.", &[" "]), + ]; + + let (updates, lines) = futures::executor::block_on(crate::test_support::logs::logged_at( + log::Level::Debug, + async { collect_eid_updates(&module_eids, None, None, ®istry) }, + )); + + assert!(updates.is_empty(), "neither EID should give an update"); + assert_eq!( + lines.len(), + 2, + "each skipped EID should be logged: {lines:?}" + ); + for sent in ["reader-sent.example", "ID5-Sync.COM."] { + assert!( + lines.iter().all(|line| !line.contains(sent)), + "no line should carry the source `{sent}`: {lines:?}" + ); + } + assert!( + lines[1].contains("'id5-sync.com'"), + "the partner's EID should be named by its configured domain: {lines:?}" + ); + } + + #[test] + fn collect_eid_updates_uses_later_valid_uid_candidate() { + let registry = make_registry(&["id5-sync.com"]); + let oversized = "x".repeat(MAX_UID_LENGTH + 1); + let module_eids = vec![eid("id5-sync.com", &[&oversized, "ID5_valid"])]; + + let updates = collect_eid_updates(&module_eids, None, None, ®istry); + + assert_eq!( + updates, + vec![PartnerIdUpdate::new("id5-sync.com", "ID5_valid")] + ); + } + + #[test] + fn collect_sharedid_update_maps_configured_sharedid_partner() { + let registry = make_registry(&["sharedid.org"]); + + let update = collect_sharedid_update(" shared-cookie-id ", ®istry) + .expect("should collect sharedId update"); + + assert_eq!( + update, + PartnerIdUpdate::new("sharedid.org", "shared-cookie-id") + ); + } + + #[test] + fn collect_eid_updates_merges_module_eids_and_sharedid_without_kv() { + let registry = make_registry(&["id5-sync.com", "sharedid.org"]); + let module_eids = vec![eid("id5-sync.com", &["ID5_abc"])]; + + let updates = collect_eid_updates(&module_eids, Some(" shared-1 "), None, ®istry); + + assert_eq!( + updates.len(), + 2, + "should collect module and sharedId matches" + ); + assert!(updates.contains(&PartnerIdUpdate::new("id5-sync.com", "ID5_abc"))); + assert!(updates.contains(&PartnerIdUpdate::new("sharedid.org", "shared-1"))); + } + + #[test] + fn collect_eid_updates_body_eids_ingest_partners_a_module_trimmed() { + // A module's cookie the browser trimmed to fit its size cap dropped + // the `liveramp.com` source, while the `/auction` request body still + // carried it, and the newer `id5-sync.com` uid, in full. + let registry = make_registry(&["id5-sync.com", "liveramp.com"]); + let module_eids = vec![eid("id5-sync.com", &["ID5_stale"])]; + let body_eids = vec![ + eid("id5-sync.com", &["ID5_fresh"]), + eid("liveramp.com", &["LR_xyz"]), + ]; + + let updates = collect_eid_updates(&module_eids, None, Some(&body_eids), ®istry); + + assert_eq!( + updates, + vec![ + PartnerIdUpdate::new("id5-sync.com", "ID5_fresh"), + PartnerIdUpdate::new("liveramp.com", "LR_xyz"), + ], + "body EIDs should ingest every configured partner present in the \ + request, including the one the module's cookie trimmed, and should \ + win over a stale value for a partner both carry" + ); + } + + #[test] + fn collect_eid_updates_empty_registry_returns_no_updates() { + let registry = PartnerRegistry::empty(); + let module_eids = vec![eid("id5-sync.com", &["ID5_abc"])]; + + let updates = collect_eid_updates(&module_eids, Some("shared-1"), None, ®istry); + + assert!( + updates.is_empty(), + "an empty registry matches no partners and touches no KV" + ); + } + + #[test] + fn dedupe_partner_updates_uses_last_partner_value() { + let updates = vec![ + PartnerIdUpdate::new("sharedid.org", "module-shared"), + PartnerIdUpdate::new("id5-sync.com", "id5-uid"), + PartnerIdUpdate::new("sharedid.org", "cookie-shared"), + ]; + + let deduped = dedupe_partner_updates(updates); + + assert_eq!(deduped.len(), 2, "should keep one update per partner"); + assert_eq!( + deduped, + vec![ + PartnerIdUpdate::new("id5-sync.com", "id5-uid"), + PartnerIdUpdate::new("sharedid.org", "cookie-shared"), + ], + "should keep the last value for duplicate partners" + ); + } + + #[test] + fn collect_eid_updates_combines_sources() { + let registry = make_registry(&["id5-sync.com", "liveramp.com", "sharedid.org"]); + let module_eids = vec![ + eid("id5-sync.com", &["ID5_abc"]), + eid("liveramp.com", &["LR_xyz"]), + ]; + + let updates = collect_eid_updates(&module_eids, Some("shared-cookie-id"), None, ®istry); + + assert_eq!( + updates, + vec![ + PartnerIdUpdate::new("id5-sync.com", "ID5_abc"), + PartnerIdUpdate::new("liveramp.com", "LR_xyz"), + PartnerIdUpdate::new("sharedid.org", "shared-cookie-id"), + ], + "should collect every matched update in one batch" + ); + } + + #[test] + fn collect_eid_updates_preserves_an_opaque_envelope() { + let registry = make_registry(&["liveramp.com"]); + let module_eids = vec![eid("liveramp.com", &["opaque-test-envelope"])]; + + let updates = collect_eid_updates(&module_eids, None, None, ®istry); + + assert_eq!( + updates, + vec![PartnerIdUpdate::new("liveramp.com", "opaque-test-envelope")], + "should preserve the opaque envelope without decoding it" + ); + } + + #[test] + fn collect_eid_updates_prefers_direct_sharedid_cookie() { + let registry = make_registry(&["sharedid.org"]); + let module_eids = vec![eid("sharedid.org", &["module-shared"])]; + + let updates = collect_eid_updates(&module_eids, Some("cookie-shared"), None, ®istry); + + assert_eq!( + updates, + vec![PartnerIdUpdate::new("sharedid.org", "cookie-shared")], + "should apply the direct sharedId cookie after the modules' EIDs" + ); + } + + #[test] + fn collect_eid_updates_ignores_unknown_sources() { + let registry = make_registry(&["id5-sync.com"]); + let module_eids = vec![eid("unknown.example", &["unknown"])]; + + let updates = collect_eid_updates(&module_eids, None, None, ®istry); + + assert!( + updates.is_empty(), + "should ignore unmatched partner updates" + ); + } +} diff --git a/crates/trusted-server-core/src/ec/mod.rs b/crates/trusted-server-core/src/ec/mod.rs index 0b46d3bb45..3f6d72a7f0 100644 --- a/crates/trusted-server-core/src/ec/mod.rs +++ b/crates/trusted-server-core/src/ec/mod.rs @@ -29,6 +29,8 @@ //! - [`rate_limiter`] — Rate limiting abstraction (implemented by adapters) //! - [`identify`] — Identity read endpoint (`GET /_ts/api/v1/identify`) //! - [`eids`] — Shared EID resolution and formatting helpers +//! - [`identifiers`]: the identifier module interface, and the sync of what +//! a module learned into the identity graph //! - [`batch_sync`] — S2S batch sync endpoint (`POST /_ts/api/v1/batch-sync`) //! - [`pull_sync`] — Background pull-sync dispatcher for organic routes @@ -41,13 +43,13 @@ pub mod device; pub mod eids; pub mod finalize; pub mod generation; +pub mod identifiers; pub mod identify; pub mod kv; pub mod kv_backend; pub mod kv_types; pub mod module; pub mod partner; -pub mod prebid_eids; pub mod pull_sync; pub(crate) mod pull_sync_marker; pub mod rate_limiter; @@ -318,11 +320,17 @@ pub struct EcContext { /// and writes, from the request's services. public_prefix: PublicPrefix, /// EIDs the route resolved from the current request: the `/auction` JSON - /// body, or the `ts-eids` cookie when the body carried none. Carried to - /// response finalization so KV ingestion can use the full set the client - /// sent instead of being limited to whatever fits in the size-capped - /// `ts-eids` cookie. + /// body, or what the identifier modules learned when the body carried + /// none. Carried to response finalization so KV ingestion can use the + /// full set the client sent instead of being limited to whatever fits in + /// a module's size-capped cookie. client_eids: Option>, + /// Identifiers the identifier modules learned for the request, asked + /// once when the state was read, where the request's evidence is in + /// hand. Carried to response finalization, which offers them to the + /// identity graph, and to the auction, which forwards them when the + /// request body carries none. + module_eids: Vec, /// Browser-carried proof of recent pull-partner completeness. pull_sync_marker: PullSyncMarkerState, /// Allowed returning-user EID persistence source, assigned only after request filters pass. @@ -526,6 +534,19 @@ impl EcContext { geo_status, services.permission_signal_modules(), ); + // Ask the identifier modules once, here, where the request's evidence + // is in hand, and carry the answer to finalization and the auction, + // which run after routing has consumed the request. With no Edge + // Cookie module nothing can hold or forward an identifier, so none + // are asked. + let module_eids = if selected_module.is_some() { + identifiers::learn( + &module_context.with_permissions(&permissions), + services.identifier_modules(), + ) + } else { + Vec::new() + }; // A signal no module vouched for goes no further. The module for // its scheme has already decided what its absence means, so what is // forwarded is exactly what the permissions were built from. @@ -564,6 +585,7 @@ impl EcContext { recovery_eligible: false, public_prefix: services.public_prefix().clone(), client_eids: None, + module_eids, pull_sync_marker: PullSyncMarkerState::from_cookie(parsed.pull_sync_marker), eid_sync_source: None, }) @@ -1038,6 +1060,18 @@ impl EcContext { self.client_eids.as_deref() } + /// The identifiers the identifier modules learned for the request. + #[must_use] + pub fn module_eids(&self) -> &[Eid] { + &self.module_eids + } + + /// States what the identifier modules learned, in place of asking them. + #[cfg(test)] + pub(crate) fn set_module_eids_for_test(&mut self, eids: Vec) { + self.module_eids = eids; + } + /// Validates a browser completeness marker against the active EC and partner set. pub(crate) fn validate_pull_sync_marker( &mut self, @@ -1219,6 +1253,7 @@ impl EcContext { recovery_eligible: false, public_prefix: PublicPrefix::default(), client_eids: None, + module_eids: Vec::new(), pull_sync_marker: PullSyncMarkerState::Absent, eid_sync_source: None, } @@ -1266,6 +1301,7 @@ impl EcContext { recovery_eligible: false, public_prefix: PublicPrefix::default(), client_eids: None, + module_eids: Vec::new(), pull_sync_marker: PullSyncMarkerState::Absent, eid_sync_source: None, } @@ -1302,6 +1338,7 @@ impl EcContext { recovery_eligible: false, public_prefix: PublicPrefix::default(), client_eids: None, + module_eids: Vec::new(), pull_sync_marker: PullSyncMarkerState::Absent, eid_sync_source: None, } @@ -2402,7 +2439,6 @@ pub(crate) mod tests { Some(&graph), ®istry::PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -2464,7 +2500,6 @@ pub(crate) mod tests { Some(&graph), ®istry::PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -2509,7 +2544,6 @@ pub(crate) mod tests { Some(graph), ®istry::PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) @@ -2604,7 +2638,6 @@ pub(crate) mod tests { Some(&graph), ®istry::PartnerRegistry::empty(), None, - None, &mut response, &noop_services(), ) diff --git a/crates/trusted-server-core/src/ec/module.rs b/crates/trusted-server-core/src/ec/module.rs index 496bd4f133..0f035d82f9 100644 --- a/crates/trusted-server-core/src/ec/module.rs +++ b/crates/trusted-server-core/src/ec/module.rs @@ -1496,7 +1496,7 @@ mod tests { ), ( "Set-Cookie", - " TS-EIDS=x; Path=/", + " TS-ECR=x; Path=/", ReservedResponseEffect::ManagedCookie, ), ("x-ts-ec", "spoofed", ReservedResponseEffect::ReservedHeader), diff --git a/crates/trusted-server-core/src/ec/prebid_eids.rs b/crates/trusted-server-core/src/ec/prebid_eids.rs deleted file mode 100644 index 3f72fb620e..0000000000 --- a/crates/trusted-server-core/src/ec/prebid_eids.rs +++ /dev/null @@ -1,733 +0,0 @@ -//! Prebid EID cookie ingestion. -//! -//! Parses the `ts-eids` cookie written by the TSJS Prebid integration and -//! syncs matched partner UIDs to the KV identity graph. -//! -//! The current cookie format stores a base64-encoded JSON array of full -//! OpenRTB-style `Eid` objects (`{source, uids:[...]}`). For rollout -//! compatibility we also accept the earlier flattened payload shape -//! (`{source, id, atype}` per entry). - -use base64::{Engine as _, engine::general_purpose::STANDARD as BASE64}; -use serde::Deserialize; -use serde_json::Value as JsonValue; - -use crate::openrtb::{Eid, Uid}; - -use super::kv::PartnerIdUpdate; -use super::kv_types::MAX_UID_LENGTH; -use super::registry::PartnerRegistry; - -/// Maximum raw `ts-eids` cookie size accepted before base64 decode. -const MAX_EIDS_COOKIE_BYTES: usize = 8 * 1024; - -/// Legacy flattened `ts-eids` cookie entry. -#[derive(Debug, Deserialize)] -struct LegacyCookieEid { - source: String, - id: String, - atype: i32, -} - -/// OpenRTB-style `ts-eids` cookie entry. -#[derive(Debug, Deserialize)] -struct StructuredCookieEid { - source: String, - #[serde(default)] - uids: Vec, -} - -#[derive(Debug, Deserialize)] -struct StructuredCookieUid { - id: String, - #[serde(default)] - atype: Option, - #[serde(default)] - ext: Option, -} - -enum DecodedCookieEids { - Legacy(Vec), - Structured(Vec), -} - -/// Parses a `ts-eids` cookie value into OpenRTB-style `Eid` entries. -/// -/// Accepts both the current structured cookie format and the earlier legacy -/// flattened format for backwards compatibility. -/// -/// # Errors -/// -/// Returns an error when the cookie exceeds the raw size limit, is not valid -/// base64, or does not contain either supported JSON payload shape. -pub fn parse_prebid_eids_cookie(cookie_value: &str) -> Result, String> { - decode_prebid_eids_cookie(cookie_value).map(DecodedCookieEids::into_openrtb) -} - -fn decode_prebid_eids_cookie(cookie_value: &str) -> Result { - if eids_cookie_exceeds_size_limit(cookie_value) { - return Err(format!( - "ts-eids cookie too large ({} bytes)", - cookie_value.len() - )); - } - - let bytes = BASE64 - .decode(cookie_value) - .map_err(|e| format!("base64 decode failed: {e}"))?; - - if let Ok(eids) = serde_json::from_slice::>(&bytes) { - return Ok(DecodedCookieEids::Legacy(eids)); - } - - let structured = serde_json::from_slice::>(&bytes) - .map_err(|e| format!("JSON parse failed: {e}"))?; - Ok(DecodedCookieEids::Structured(structured)) -} - -impl DecodedCookieEids { - fn into_openrtb(self) -> Vec { - match self { - Self::Legacy(entries) => legacy_cookie_eids_to_openrtb(entries), - Self::Structured(entries) => structured_cookie_eids_to_openrtb(entries), - } - } -} - -/// Collects validated request-local partner updates without performing KV I/O. -/// -/// `body_eids` is the current request's own EIDs (e.g. an `/auction` JSON -/// body), when the route captured any. It is applied after the `ts-eids` -/// cookie so it wins on conflicts: the cookie is size-capped by the browser -/// (see `MAX_EID_COOKIE_BYTES` in the TSJS Prebid integration) and may be -/// missing partners the request body still carries in full. The `sharedId` -/// cookie is applied last, unaffected by `body_eids`, preserving its existing -/// override behavior. -pub(crate) fn collect_eid_updates( - eids_cookie: Option<&str>, - sharedid_cookie: Option<&str>, - body_eids: Option<&[Eid]>, - registry: &PartnerRegistry, -) -> Vec { - if registry.is_empty() { - return Vec::new(); - } - - let mut updates = Vec::new(); - if let Some(cookie) = eids_cookie { - updates.extend(collect_prebid_eid_updates(cookie, registry)); - } - if let Some(eids) = body_eids { - updates.extend(collect_prebid_eid_updates_from_eids(eids, registry)); - } - if let Some(cookie) = sharedid_cookie - && let Some(update) = collect_sharedid_update(cookie, registry) - { - updates.push(update); - } - dedupe_partner_updates(updates) -} - -pub(crate) fn collect_prebid_eid_updates( - cookie_value: &str, - registry: &PartnerRegistry, -) -> Vec { - let Ok(eids) = parse_prebid_eids_cookie(cookie_value) else { - log::trace!("Prebid EIDs: failed to decode ts-eids cookie; dropping"); - return Vec::new(); - }; - - collect_prebid_eid_updates_from_eids(&eids, registry) -} - -fn collect_prebid_eid_updates_from_eids( - eids: &[Eid], - registry: &PartnerRegistry, -) -> Vec { - let mut updates = Vec::new(); - for eid in eids { - let Some(partner) = registry.find_by_source_domain(&eid.source) else { - log::debug!("Prebid EIDs: no partner for source '{}'", eid.source); - continue; - }; - - // KV stores one UID per partner. Preserve the previous cookie-ingestion - // behavior by syncing the first valid UID under each source, while - // skipping malformed candidates instead of dropping the whole source. - let Some(uid) = first_valid_uid(&eid.uids) else { - log::debug!( - "Prebid EIDs: no valid uid for source_domain '{}' from source '{}'", - partner.source_domain, - eid.source, - ); - continue; - }; - - updates.push(PartnerIdUpdate::new(&partner.source_domain, &uid.id)); - } - - updates -} - -fn dedupe_partner_updates(updates: Vec) -> Vec { - let mut latest = std::collections::BTreeMap::new(); - for update in updates { - latest.insert(update.partner_id, update.uid); - } - - latest - .into_iter() - .map(|(partner_id, uid)| PartnerIdUpdate::new(partner_id, uid)) - .collect() -} - -fn first_valid_uid(uids: &[Uid]) -> Option<&Uid> { - uids.iter().find(|uid| is_valid_eid_uid(&uid.id)) -} - -fn is_valid_eid_uid(uid: &str) -> bool { - !uid.trim().is_empty() && !eid_id_exceeds_size_limit(uid) -} - -/// `SharedID` EID source domain used for partner registry lookup. -const SHAREDID_SOURCE_DOMAIN: &str = "sharedid.org"; - -fn collect_sharedid_update( - cookie_value: &str, - registry: &PartnerRegistry, -) -> Option { - let cookie_value = cookie_value.trim(); - if cookie_value.is_empty() { - return None; - } - - if sharedid_cookie_exceeds_size_limit(cookie_value) { - log::debug!( - "SharedID: cookie exceeds MAX_UID_LENGTH ({} bytes)", - cookie_value.len() - ); - return None; - } - - let Some(partner) = registry.find_by_source_domain(SHAREDID_SOURCE_DOMAIN) else { - log::debug!("SharedID: no partner configured for source '{SHAREDID_SOURCE_DOMAIN}'"); - return None; - }; - - Some(PartnerIdUpdate::new(&partner.source_domain, cookie_value)) -} - -fn eids_cookie_exceeds_size_limit(cookie_value: &str) -> bool { - cookie_value.len() > MAX_EIDS_COOKIE_BYTES -} - -fn eid_id_exceeds_size_limit(uid: &str) -> bool { - uid.len() > MAX_UID_LENGTH -} - -fn sharedid_cookie_exceeds_size_limit(cookie_value: &str) -> bool { - cookie_value.len() > MAX_UID_LENGTH -} - -fn structured_cookie_eids_to_openrtb(entries: Vec) -> Vec { - let mut eids = Vec::new(); - - for entry in entries { - if entry.source.is_empty() { - continue; - } - - let uids: Vec<_> = entry - .uids - .into_iter() - .filter_map(structured_cookie_uid_to_openrtb) - .collect(); - if uids.is_empty() { - continue; - } - - eids.push(Eid { - source: entry.source, - uids, - }); - } - - eids -} - -fn structured_cookie_uid_to_openrtb(uid: StructuredCookieUid) -> Option { - if uid.id.is_empty() { - return None; - } - - let atype = uid.atype.filter(|atype| *atype >= 0); - let ext = match uid.ext { - Some(JsonValue::Object(_)) => uid.ext, - _ => None, - }; - - Some(Uid { - id: uid.id, - atype, - ext, - }) -} - -fn legacy_cookie_eids_to_openrtb(entries: Vec) -> Vec { - entries - .into_iter() - .filter(|entry| !entry.source.is_empty() && !entry.id.is_empty()) - .map(|entry| Eid { - source: entry.source, - uids: vec![Uid { - id: entry.id, - atype: (entry.atype >= 0).then_some(entry.atype), - ext: None, - }], - }) - .collect() -} - -#[cfg(test)] -mod tests { - use super::*; - use base64::engine::general_purpose::STANDARD as BASE64; - use serde_json::json; - - use crate::ec::registry::PartnerRegistry; - use crate::redacted::Redacted; - use crate::settings::EcPartner; - - fn make_test_partner(_id: &str, source_domain: &str) -> EcPartner { - EcPartner { - name: format!("Partner {source_domain}"), - source_domain: source_domain.to_owned(), - openrtb_atype: EcPartner::default_openrtb_atype(), - bidstream_enabled: true, - api_token: Some(Redacted::new(format!( - "token-{source_domain}-32-bytes-minimum-value" - ))), - batch_rate_limit: EcPartner::default_batch_rate_limit(), - pull_sync_enabled: false, - pull_sync_url: None, - pull_sync_allowed_domains: vec![], - pull_sync_ttl_sec: EcPartner::default_pull_sync_ttl_sec(), - pull_sync_rate_limit: EcPartner::default_pull_sync_rate_limit(), - ts_pull_token: None, - } - } - - fn make_registry(partners: Vec<(&str, &str)>) -> PartnerRegistry { - let partners: Vec<_> = partners - .into_iter() - .map(|(id, source_domain)| make_test_partner(id, source_domain)) - .collect(); - PartnerRegistry::from_config(&partners).expect("should build partner registry") - } - - fn encode_json(value: &serde_json::Value) -> String { - BASE64.encode(serde_json::to_vec(value).expect("should serialize EID JSON")) - } - - #[test] - fn parse_prebid_eids_cookie_parses_legacy_flat_payload() { - let eids = vec![ - json!({"source": "id5-sync.com", "id": "ID5_abc", "atype": 1}), - json!({"source": "liveramp.com", "id": "LR_xyz", "atype": 3}), - json!({"source": "google.com", "id": "pair-id", "atype": 571187}), - ]; - let encoded = BASE64.encode(serde_json::to_vec(&eids).expect("should serialize")); - - let decoded = parse_prebid_eids_cookie(&encoded).expect("should decode valid payload"); - assert_eq!(decoded.len(), 3, "should parse all EIDs"); - assert_eq!(decoded[0].source, "id5-sync.com"); - assert_eq!(decoded[0].uids[0].id, "ID5_abc"); - assert_eq!(decoded[1].source, "liveramp.com"); - assert_eq!(decoded[1].uids[0].id, "LR_xyz"); - assert_eq!( - decoded[2].source, "google.com", - "should preserve PAIR source" - ); - assert_eq!( - decoded[2].uids[0].atype, - Some(571187), - "should preserve PAIR vendor-specific atype" - ); - } - - #[test] - fn parse_prebid_eids_cookie_parses_structured_payload() { - let eids = vec![json!({ - "source": "sharedid.org", - "uids": [ - {"id": "shared_123", "atype": 3}, - {"id": "shared_456", "ext": {"provider": "example"}}, - {"id": "shared_invalid", "atype": -1} - ] - })]; - let encoded = BASE64.encode(serde_json::to_vec(&eids).expect("should serialize")); - - let decoded = parse_prebid_eids_cookie(&encoded).expect("should decode valid payload"); - assert_eq!(decoded.len(), 1, "should parse one structured EID entry"); - assert_eq!(decoded[0].source, "sharedid.org"); - assert_eq!(decoded[0].uids.len(), 3, "should preserve multiple UIDs"); - assert_eq!(decoded[0].uids[0].id, "shared_123"); - assert_eq!(decoded[0].uids[0].atype, Some(3)); - assert_eq!( - decoded[0].uids[1].ext, - Some(json!({"provider": "example"})), - "should preserve UID ext objects" - ); - assert_eq!( - decoded[0].uids[2].atype, None, - "should drop negative atype values" - ); - } - - #[test] - fn parse_prebid_eids_cookie_preserves_pair_atype() { - let encoded = encode_json(&json!([ - { - "source": "google.com", - "uids": [{ "id": "pair-id", "atype": 571187 }] - } - ])); - - let decoded = parse_prebid_eids_cookie(&encoded).expect("should decode PAIR EID"); - - assert_eq!( - decoded[0].uids[0].atype, - Some(571187), - "should preserve PAIR's vendor-specific atype" - ); - } - - #[test] - fn parse_prebid_eids_cookie_rejects_invalid_base64() { - let result = parse_prebid_eids_cookie("not-valid-base64!!!"); - assert!(result.is_err(), "should reject invalid base64"); - } - - #[test] - fn parse_prebid_eids_cookie_rejects_invalid_json() { - let encoded = BASE64.encode(b"not json"); - let result = parse_prebid_eids_cookie(&encoded); - assert!(result.is_err(), "should reject invalid JSON"); - } - - #[test] - fn ts_eids_cookie_rejects_oversized_payloads() { - let oversized = "x".repeat(MAX_EIDS_COOKIE_BYTES + 1); - let exact_limit = "x".repeat(MAX_EIDS_COOKIE_BYTES); - - assert!( - eids_cookie_exceeds_size_limit(&oversized), - "should reject cookies larger than the raw size cap" - ); - assert!( - !eids_cookie_exceeds_size_limit(&exact_limit), - "should allow cookies exactly at the raw size cap" - ); - } - - #[test] - fn sharedid_cookie_rejects_values_larger_than_uid_limit() { - let oversized = "x".repeat(MAX_UID_LENGTH + 1); - let exact_limit = "x".repeat(MAX_UID_LENGTH); - - assert!( - sharedid_cookie_exceeds_size_limit(&oversized), - "should reject sharedId values larger than MAX_UID_LENGTH" - ); - assert!( - !sharedid_cookie_exceeds_size_limit(&exact_limit), - "should allow sharedId values exactly at MAX_UID_LENGTH" - ); - } - - #[test] - fn prebid_eid_uid_rejects_values_larger_than_uid_limit() { - let oversized = "x".repeat(MAX_UID_LENGTH + 1); - let exact_limit = "x".repeat(MAX_UID_LENGTH); - - assert!( - eid_id_exceeds_size_limit(&oversized), - "should reject EID values larger than MAX_UID_LENGTH" - ); - assert!( - !eid_id_exceeds_size_limit(&exact_limit), - "should allow EID values exactly at MAX_UID_LENGTH" - ); - } - - #[test] - fn first_valid_uid_skips_oversized_and_uses_later_valid_uid() { - let oversized = "x".repeat(MAX_UID_LENGTH + 1); - let uids = vec![ - Uid { - id: oversized, - atype: Some(1), - ext: None, - }, - Uid { - id: "valid-uid".to_owned(), - atype: Some(1), - ext: None, - }, - ]; - - let uid = first_valid_uid(&uids).expect("should find later valid UID"); - assert_eq!(uid.id, "valid-uid", "should skip oversized first UID"); - } - - #[test] - fn first_valid_uid_rejects_whitespace_only_ids() { - let uids = vec![Uid { - id: " ".to_owned(), - atype: Some(1), - ext: None, - }]; - - assert!( - first_valid_uid(&uids).is_none(), - "should reject whitespace UID" - ); - } - - #[test] - fn collect_prebid_eid_updates_collects_multiple_partner_matches() { - let registry = make_registry(vec![("id5", "id5-sync.com"), ("liveramp", "liveramp.com")]); - let cookie = encode_json(&json!([ - {"source": "id5-sync.com", "uids": [{"id": "ID5_abc", "atype": 1}]}, - {"source": "liveramp.com", "uids": [{"id": "LR_xyz", "atype": 3}]} - ])); - - let updates = collect_prebid_eid_updates(&cookie, ®istry); - - assert_eq!(updates.len(), 2, "should collect both partner matches"); - assert_eq!(updates[0], PartnerIdUpdate::new("id5-sync.com", "ID5_abc")); - assert_eq!(updates[1], PartnerIdUpdate::new("liveramp.com", "LR_xyz")); - } - - #[test] - fn collect_prebid_eid_updates_skips_unknown_sources() { - let registry = make_registry(vec![("id5", "id5-sync.com")]); - let cookie = encode_json(&json!([ - {"source": "unknown.example", "uids": [{"id": "unknown", "atype": 1}]} - ])); - - let updates = collect_prebid_eid_updates(&cookie, ®istry); - - assert!( - updates.is_empty(), - "should skip EIDs without configured source-domain partners" - ); - } - - #[test] - fn collect_prebid_eid_updates_uses_later_valid_uid_candidate() { - let registry = make_registry(vec![("id5", "id5-sync.com")]); - let oversized = "x".repeat(MAX_UID_LENGTH + 1); - let cookie = encode_json(&json!([ - { - "source": "id5-sync.com", - "uids": [ - {"id": oversized, "atype": 1}, - {"id": "ID5_valid", "atype": 1} - ] - } - ])); - - let updates = collect_prebid_eid_updates(&cookie, ®istry); - - assert_eq!( - updates, - vec![PartnerIdUpdate::new("id5-sync.com", "ID5_valid")] - ); - } - - #[test] - fn collect_sharedid_update_maps_configured_sharedid_partner() { - let registry = make_registry(vec![("sharedid", "sharedid.org")]); - - let update = collect_sharedid_update(" shared-cookie-id ", ®istry) - .expect("should collect sharedId update"); - - assert_eq!( - update, - PartnerIdUpdate::new("sharedid.org", "shared-cookie-id") - ); - } - - #[test] - fn collect_eid_updates_merges_prebid_and_sharedid_without_kv() { - let registry = make_registry(vec![("id5", "id5-sync.com"), ("sharedid", "sharedid.org")]); - let eids_cookie = encode_json(&json!([ - {"source": "id5-sync.com", "uids": [{"id": "ID5_abc", "atype": 1}]} - ])); - - let updates = collect_eid_updates(Some(&eids_cookie), Some(" shared-1 "), None, ®istry); - - assert_eq!( - updates.len(), - 2, - "should collect prebid and sharedId matches" - ); - assert!(updates.contains(&PartnerIdUpdate::new("id5-sync.com", "ID5_abc"))); - assert!(updates.contains(&PartnerIdUpdate::new("sharedid.org", "shared-1"))); - } - - #[test] - fn collect_eid_updates_body_eids_ingest_partners_the_cookie_trimmed() { - // Simulates a `ts-eids` cookie that the browser trimmed to fit - // `MAX_EID_COOKIE_BYTES`, dropping the `liveramp.com` source, while - // the `/auction` request body still carried it (and the newer, - // updated `id5-sync.com` uid) in full. - let registry = make_registry(vec![("id5", "id5-sync.com"), ("liveramp", "liveramp.com")]); - let eids_cookie = encode_json(&json!([ - {"source": "id5-sync.com", "uids": [{"id": "ID5_stale", "atype": 1}]} - ])); - let body_eids = vec![ - Eid { - source: "id5-sync.com".to_owned(), - uids: vec![Uid { - id: "ID5_fresh".to_owned(), - atype: Some(1), - ext: None, - }], - }, - Eid { - source: "liveramp.com".to_owned(), - uids: vec![Uid { - id: "LR_xyz".to_owned(), - atype: Some(3), - ext: None, - }], - }, - ]; - - let updates = collect_eid_updates(Some(&eids_cookie), None, Some(&body_eids), ®istry); - - assert_eq!( - updates, - vec![ - PartnerIdUpdate::new("id5-sync.com", "ID5_fresh"), - PartnerIdUpdate::new("liveramp.com", "LR_xyz"), - ], - "body EIDs should ingest every configured partner present in the \ - request, including the one the cookie trimmed, and should win \ - over a stale cookie value for a partner both carry" - ); - } - - #[test] - fn collect_eid_updates_empty_registry_returns_no_updates() { - let registry = PartnerRegistry::empty(); - let eids_cookie = encode_json(&json!([ - {"source": "id5-sync.com", "uids": [{"id": "ID5_abc", "atype": 1}]} - ])); - - let updates = collect_eid_updates(Some(&eids_cookie), Some("shared-1"), None, ®istry); - - assert!( - updates.is_empty(), - "an empty registry matches no partners and touches no KV" - ); - } - - #[test] - fn dedupe_partner_updates_uses_last_partner_value() { - let updates = vec![ - PartnerIdUpdate::new("sharedid.org", "prebid-shared"), - PartnerIdUpdate::new("id5-sync.com", "id5-uid"), - PartnerIdUpdate::new("sharedid.org", "cookie-shared"), - ]; - - let deduped = dedupe_partner_updates(updates); - - assert_eq!(deduped.len(), 2, "should keep one update per partner"); - assert_eq!( - deduped, - vec![ - PartnerIdUpdate::new("id5-sync.com", "id5-uid"), - PartnerIdUpdate::new("sharedid.org", "cookie-shared"), - ], - "should keep the last value for duplicate partners" - ); - } - - #[test] - fn collect_eid_updates_combines_cookie_sources() { - let registry = make_registry(vec![ - ("id5", "id5-sync.com"), - ("liveramp", "liveramp.com"), - ("sharedid", "sharedid.org"), - ]); - let cookie = encode_json(&json!([ - {"source": "id5-sync.com", "uids": [{"id": "ID5_abc", "atype": 1}]}, - {"source": "liveramp.com", "uids": [{"id": "LR_xyz", "atype": 3}]} - ])); - - let updates = collect_eid_updates(Some(&cookie), Some("shared-cookie-id"), None, ®istry); - - assert_eq!( - updates, - vec![ - PartnerIdUpdate::new("id5-sync.com", "ID5_abc"), - PartnerIdUpdate::new("liveramp.com", "LR_xyz"), - PartnerIdUpdate::new("sharedid.org", "shared-cookie-id"), - ], - "should collect every matched update in one batch" - ); - } - - #[test] - fn collect_liveramp_eid_cookie_updates_preserves_the_opaque_envelope() { - let registry = make_registry(vec![("liveramp", "liveramp.com")]); - let cookie = encode_json(&json!([ - { - "source": "liveramp.com", - "uids": [{"id": "opaque-test-envelope", "atype": 3}] - } - ])); - - let updates = collect_eid_updates(Some(&cookie), None, None, ®istry); - - assert_eq!( - updates, - vec![PartnerIdUpdate::new("liveramp.com", "opaque-test-envelope")], - "should preserve the opaque envelope without decoding it" - ); - } - - #[test] - fn collect_eid_updates_prefers_direct_sharedid_cookie() { - let registry = make_registry(vec![("sharedid", "sharedid.org")]); - let cookie = encode_json(&json!([ - {"source": "sharedid.org", "uids": [{"id": "prebid-shared", "atype": 3}]} - ])); - - let updates = collect_eid_updates(Some(&cookie), Some("cookie-shared"), None, ®istry); - - assert_eq!( - updates, - vec![PartnerIdUpdate::new("sharedid.org", "cookie-shared")], - "should apply the direct sharedId cookie after Prebid EIDs" - ); - } - - #[test] - fn collect_eid_updates_ignores_unknown_sources() { - let registry = make_registry(vec![("id5", "id5-sync.com")]); - let cookie = encode_json(&json!([ - {"source": "unknown.example", "uids": [{"id": "unknown", "atype": 1}]} - ])); - - let updates = collect_eid_updates(Some(&cookie), None, None, ®istry); - - assert!( - updates.is_empty(), - "should ignore unmatched partner updates" - ); - } -} diff --git a/crates/trusted-server-core/src/ec/resolve.rs b/crates/trusted-server-core/src/ec/resolve.rs index fea625016d..9934982b1e 100644 --- a/crates/trusted-server-core/src/ec/resolve.rs +++ b/crates/trusted-server-core/src/ec/resolve.rs @@ -1428,7 +1428,6 @@ mod tests { Some(graph), &PartnerRegistry::empty(), None, - None, &mut response, services, ) @@ -1459,7 +1458,6 @@ mod tests { Some(graph), &PartnerRegistry::empty(), None, - None, &mut response, services, ) diff --git a/crates/trusted-server-core/src/integrations/registry.rs b/crates/trusted-server-core/src/integrations/registry.rs index f29942b9e1..efad79e409 100644 --- a/crates/trusted-server-core/src/integrations/registry.rs +++ b/crates/trusted-server-core/src/integrations/registry.rs @@ -14,6 +14,7 @@ use crate::auction::AuctionPlan; use crate::constants::HEADER_X_TS_EC; use crate::ec::EcContext; use crate::ec::device::DeviceModule; +use crate::ec::identifiers::IdentifierModule; use crate::ec::kv::KvIdentityGraph; use crate::ec::module::{EcModuleSelection, EdgeCookieModule}; use crate::error::TrustedServerError; @@ -693,6 +694,13 @@ pub struct IntegrationRegistration { /// Declaring one does not make it serve anything, because a source is only /// built where `[source]` or an asset route's `source` table names it. pub source_module: Option<(&'static str, Arc)>, + /// Identifier module this registration supplies, see + /// [`crate::ec::identifiers`]. + /// + /// No section selects one. Core asks every running registration's module + /// on every request, so declaring one runs it wherever the registration + /// itself runs. + pub identifier_module: Option>, /// Asset cache module this registration supplies, under its name. /// /// Taken only from the registration of the builder `[asset-cache]` @@ -730,6 +738,7 @@ impl IntegrationRegistrationBuilder { prefix_module: None, robots_txt_contributor: None, source_module: None, + identifier_module: None, asset_cache_module: None, }, } @@ -837,6 +846,15 @@ impl IntegrationRegistrationBuilder { self } + /// Declare the identifier module this registration supplies, which core + /// asks for the identifiers it learned on every request the registration + /// runs for, see [`crate::ec::identifiers`]. + #[must_use] + pub fn with_identifier_module(mut self, module: Arc) -> Self { + self.registration.identifier_module = Some(module); + self + } + /// Declare the rules this registration adds to `/robots.txt`, under their /// `name`. /// @@ -994,6 +1012,10 @@ struct IntegrationRegistryInner { // The publisher's source and each asset route's, in route order, built // from the settings. `None` for a registry built without settings. sources: Option, + // The identifier modules the running registrations supply, in + // registration order, which core asks on every request. Shared, so a + // request's services take a reference count rather than a copy. + identifier_modules: Arc<[Arc]>, // The asset cache module of the builder `[asset-cache]` selects, or // `None` when it selects none. asset_cache_module: Option>, @@ -1042,6 +1064,7 @@ impl Default for IntegrationRegistryInner { prefix_module: SelectedPrefixModule::default(), source_modules: Vec::new(), sources: None, + identifier_modules: Arc::default(), asset_cache_module: None, } } @@ -2118,6 +2141,7 @@ impl IntegrationRegistry { // The geo, Edge Cookie and device module names taken so far, with the // integration that supplies each. let mut claimed = Vec::new(); + let mut identifier_modules = Vec::new(); for registration in plan_registrations.into_iter().chain(registrations) { inner .running_integration_ids @@ -2240,6 +2264,15 @@ impl IntegrationRegistry { claim_module_name(&mut claimed, "source", name, registration.integration_id)?; inner.source_modules.push((name, factory)); } + if let Some(module) = registration.identifier_module { + claim_module_name( + &mut claimed, + "identifier", + module.id(), + registration.integration_id, + )?; + identifier_modules.push(module); + } if let Some((name, contributor)) = registration.robots_txt_contributor { claim_module_name( &mut claimed, @@ -2327,6 +2360,7 @@ impl IntegrationRegistry { inner.device_module = resolve_device_module(settings, &inner)?; inner.prefix_module = resolve_prefix_module(settings, &inner)?; inner.sources = Some(build_sources(settings, &inner.source_modules)?); + inner.identifier_modules = Arc::from(identifier_modules); inner.geo_module = geo_module; check_robots_txt_selection(settings, &inner)?; if let Some(config) = settings.robots_txt.as_ref() { @@ -2399,6 +2433,14 @@ impl IntegrationRegistry { &self.inner.prefix_module } + /// The identifier modules the running registrations supply, in + /// registration order, which core asks on every request. Shared, so a + /// request's services take a reference count rather than a copy. + #[must_use] + pub fn identifier_modules(&self) -> Arc<[Arc]> { + Arc::clone(&self.inner.identifier_modules) + } + /// The prefixes of the publisher `request` was resolved to under /// `settings`, from the selected prefix module. /// @@ -3003,6 +3045,7 @@ impl IntegrationRegistry { prefix_module: SelectedPrefixModule::default(), source_modules: Vec::new(), sources: None, + identifier_modules: Arc::default(), asset_cache_module: None, }), plan: None, @@ -3088,6 +3131,7 @@ impl IntegrationRegistry { prefix_module: SelectedPrefixModule::default(), source_modules: Vec::new(), sources: None, + identifier_modules: Arc::default(), asset_cache_module: None, }), plan: None, @@ -7526,6 +7570,154 @@ mod tests { ); } + /// Identifier modules that learn nothing, each under a fixed name as + /// every module has. + #[derive(Debug)] + enum IdleIdentifiers { + Probe, + Second, + } + + impl crate::ec::identifiers::IdentifierModule for IdleIdentifiers { + fn id(&self) -> &'static str { + match self { + Self::Probe => IDENTIFIER_PROBE_MODULE, + Self::Second => IDENTIFIER_SECOND_MODULE, + } + } + + fn identifiers( + &self, + _call: crate::module_context::ModuleCall<'_>, + ) -> Result, crate::module_context::Withheld> { + Ok(Vec::new()) + } + } + + /// The names the two identifier test modules run under. + const IDENTIFIER_PROBE_MODULE: &str = "testing.identifier-probe"; + const IDENTIFIER_SECOND_MODULE: &str = "testing.identifier-second"; + + fn identifier_probe_registration( + _settings: &Settings, + ) -> Result, Report> { + Ok(Some( + IntegrationRegistration::builder("identifier-probe") + .with_identifier_module(Arc::new(IdleIdentifiers::Probe)) + .build(), + )) + } + + fn identifier_second_registration( + _settings: &Settings, + ) -> Result, Report> { + Ok(Some( + IntegrationRegistration::builder("identifier-second") + .with_identifier_module(Arc::new(IdleIdentifiers::Second)) + .build(), + )) + } + + /// A second registration declaring its identifier module under the + /// first's name. + fn identifier_same_name_registration( + _settings: &Settings, + ) -> Result, Report> { + Ok(Some( + IntegrationRegistration::builder("identifier-second") + .with_identifier_module(Arc::new(IdleIdentifiers::Probe)) + .build(), + )) + } + + fn identifier_probe_builder() -> crate::integrations::IntegrationBuilder { + crate::integrations::IntegrationBuilder::new( + "identifier-probe", + "seam-probe", + identifier_probe_registration, + validate_nothing, + ) + .with_module_name(IDENTIFIER_PROBE_MODULE) + } + + /// No section selects an identifier module. Every running registration's + /// is handed out, in registration order, for core to ask on each request. + #[test] + fn identifier_modules_are_handed_out_in_registration_order() { + let mut settings = settings_naming(IDENTIFIER_PROBE_MODULE); + settings.select_module("testing", IDENTIFIER_SECOND_MODULE); + let extra = [ + identifier_probe_builder(), + crate::integrations::IntegrationBuilder::new( + "identifier-second", + "seam-probe", + identifier_second_registration, + validate_nothing, + ) + .with_module_name(IDENTIFIER_SECOND_MODULE), + ]; + + let registry = IntegrationRegistry::with_registrations(&settings, &extra) + .expect("should build a registry with two identifier modules"); + + let ids: Vec<_> = registry + .identifier_modules() + .iter() + .map(|module| module.id()) + .collect(); + assert_eq!( + ids, + vec![IDENTIFIER_PROBE_MODULE, IDENTIFIER_SECOND_MODULE], + "should hand out every running registration's module in order" + ); + } + + /// A registration a section does not select runs nothing, its identifier + /// module included. + #[test] + fn an_unselected_registration_offers_no_identifier_module() { + let settings = crate::test_support::tests::create_test_settings(); + + let registry = + IntegrationRegistry::with_registrations(&settings, &[identifier_probe_builder()]) + .expect("should build a registry without the probe selected"); + + assert!( + registry.identifier_modules().is_empty(), + "an unselected registration's identifier module should not be asked" + ); + } + + /// Two registrations declaring an identifier module under one name would + /// leave a log unable to say which answered, so the pair is refused. + #[test] + fn two_registrations_declaring_one_identifier_module_name_are_refused() { + let mut settings = settings_naming(IDENTIFIER_PROBE_MODULE); + settings.select_module("testing", IDENTIFIER_SECOND_MODULE); + let extra = [ + identifier_probe_builder(), + crate::integrations::IntegrationBuilder::new( + "identifier-second", + "seam-probe", + identifier_same_name_registration, + validate_nothing, + ) + .with_module_name(IDENTIFIER_SECOND_MODULE), + ]; + + let error = IntegrationRegistry::with_registrations(&settings, &extra) + .err() + .expect("should refuse one identifier module name declared twice"); + + let message = error.to_string(); + assert!( + message.contains("the identifier module `testing.identifier-probe` is declared twice") + && message.contains("`identifier-probe`") + && message.contains("`identifier-second`"), + "error should name the module and both integrations: {message}" + ); + } + fn html_entry(path: Option<&str>, names: &[&str]) -> PhaseEntry { PhaseEntry { media_type: HTML_MEDIA_TYPE.to_owned(), diff --git a/crates/trusted-server-core/src/platform/types.rs b/crates/trusted-server-core/src/platform/types.rs index d647158a53..4dc09e08e0 100644 --- a/crates/trusted-server-core/src/platform/types.rs +++ b/crates/trusted-server-core/src/platform/types.rs @@ -11,6 +11,7 @@ use super::{ }; use crate::asset_cache::AssetCacheModule; use crate::ec::device::DeviceModule; +use crate::ec::identifiers::IdentifierModule; use crate::ec::module::EdgeCookieModule; use crate::evidence::HostSignals; use crate::permission_signal::PermissionSignalModule; @@ -229,6 +230,11 @@ pub struct RuntimeServices { /// so building the services for a request bumps a reference count rather /// than copying the list, and cloning the services does the same. pub(crate) permission_signal_modules: Arc<[Arc]>, + /// The identifier modules the running registrations supply, asked once + /// per request for the identifiers each learned, see + /// [`crate::ec::identifiers`]. Empty when none is linked. Shared, as the + /// permission signal modules are. + pub(crate) identifier_modules: Arc<[Arc]>, /// The prefixes of the publisher this request was resolved to, from the /// selected prefix module, which start every name the browser holds and /// every key in a shared store. See [`crate::prefix`]. @@ -392,6 +398,12 @@ impl RuntimeServices { &self.permission_signal_modules } + /// The identifier modules the running registrations supply, in order. + #[must_use] + pub fn identifier_modules(&self) -> &[Arc] { + &self.identifier_modules + } + /// The device module a module supplied, when `[device] module` selected /// one. `None` leaves core's own device classification in place. #[must_use] @@ -573,6 +585,17 @@ impl RuntimeServices { Self { prefixes, ..self } } + /// Returns a clone of this instance carrying the identifier modules the + /// running registrations supply, through + /// [`IntegrationRegistry::identifier_modules`](crate::integrations::IntegrationRegistry::identifier_modules). + #[must_use] + pub fn with_identifier_modules(self, modules: Arc<[Arc]>) -> Self { + Self { + identifier_modules: modules, + ..self + } + } + /// Returns a clone of this instance with the loading caches replaced. #[must_use] pub fn with_loading_caches(self, caches: Arc) -> Self { @@ -655,6 +678,7 @@ pub struct RuntimeServicesBuilder { device_module: Option>, prefixes: Option, permission_signal_modules: Arc<[Arc]>, + identifier_modules: Arc<[Arc]>, permission_policy: Option>, } @@ -678,6 +702,7 @@ impl RuntimeServicesBuilder { device_module: None, prefixes: None, permission_signal_modules: Arc::default(), + identifier_modules: Arc::default(), permission_policy: None, } } @@ -814,6 +839,18 @@ impl RuntimeServicesBuilder { self } + /// Set the identifier modules the running registrations supply, which + /// core asks once per request. + /// + /// Optional, and empty when unset. An adapter hands in what + /// [`IntegrationRegistry::identifier_modules`](crate::integrations::IntegrationRegistry::identifier_modules) + /// gives. + #[must_use] + pub fn identifier_modules(mut self, modules: Arc<[Arc]>) -> Self { + self.identifier_modules = modules; + self + } + /// Set the prefixes of the publisher the request was resolved to. /// /// Optional in the builder, because services built for no request, such @@ -887,6 +924,7 @@ impl RuntimeServicesBuilder { asset_cache_module: None, prefixes: self.prefixes.unwrap_or_default(), permission_signal_modules: self.permission_signal_modules, + identifier_modules: self.identifier_modules, permission_policy: self .permission_policy .expect("should set permission_policy before building RuntimeServices"), diff --git a/crates/trusted-server-core/src/prefix.rs b/crates/trusted-server-core/src/prefix.rs index de7b73fa41..9fd5112959 100644 --- a/crates/trusted-server-core/src/prefix.rs +++ b/crates/trusted-server-core/src/prefix.rs @@ -368,12 +368,7 @@ mod tests { assert_eq!(prefixes.public().as_str(), "ts"); assert_eq!(prefixes.server_key().as_str(), "ts.example.com"); - for (name, cookie) in [ - ("ec", "ts-ec"), - ("ecr", "ts-ecr"), - ("eids", "ts-eids"), - ("tester", "ts-tester"), - ] { + for (name, cookie) in [("ec", "ts-ec"), ("ecr", "ts-ecr"), ("tester", "ts-tester")] { assert_eq!(prefixes.public().cookie_name(name), cookie); } } @@ -412,7 +407,7 @@ mod tests { fn a_prefix_owns_the_cookie_names_it_gives_and_no_others() { let prefix = TrustedServerPrefix::public(); - for owned in ["ts-ec", "TS-EIDS", "ts-anything", "ts-"] { + for owned in ["ts-ec", "TS-ECR", "ts-anything", "ts-"] { assert!(prefix.owns_cookie_name(owned.as_bytes()), "{owned}"); } for other in ["ts", "tsx-ec", "_ex-ec", "sharedId"] { diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index d4eae85731..a44334cabe 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -27,7 +27,6 @@ use std::time::{Duration, SystemTime}; use brotli::Decompressor; use brotli::enc::BrotliEncoderParams; use brotli::enc::writer::CompressorWriter; -use cookie::CookieJar; use edgezero_core::body::Body as EdgeBody; use error_stack::{Report, ResultExt}; use flate2::read::ZlibDecoder; @@ -58,7 +57,7 @@ use crate::cache_policy::{ CachePolicy, EdgeCacheHeader, cache_control_headers_are_private_or_no_store, }; use crate::consent::{consent_allows_server_side_auction, gate_eids_by_permissions}; -use crate::constants::{COOKIE_EIDS, COOKIE_SHAREDID, HEADER_X_COMPRESS_HINT, HOST_SIGNAL_HEADERS}; +use crate::constants::{COOKIE_SHAREDID, HEADER_X_COMPRESS_HINT, HOST_SIGNAL_HEADERS}; use crate::cookies::handle_request_cookies; use crate::cookies::template_cache_policy::{TemplateCookieDecision, evaluate_cookie_policy}; use crate::creative_opportunities::{ @@ -3808,7 +3807,7 @@ struct EcSnapshotPreloadInput { has_kv: bool, marker_valid: bool, auction_needs_row: bool, - eid_cookie_may_need_persistence: bool, + eids_may_need_persistence: bool, privacy_needs_row: bool, } @@ -3816,7 +3815,7 @@ fn should_preload_ec_snapshot(input: &EcSnapshotPreloadInput) -> bool { let eligible = input.is_navigation && input.is_get && input.has_ec_id && input.has_kv; let marker_can_skip = input.marker_valid && !input.auction_needs_row - && !input.eid_cookie_may_need_persistence + && !input.eids_may_need_persistence && !input.privacy_needs_row; eligible && !marker_can_skip } @@ -5597,9 +5596,9 @@ pub async fn handle_publisher_request( }); // A key cookie naming one of this request's own identity cookies would // make the template one reader's, so such a request shares none. - let identity = - crate::cookies::template_cache_policy::identity_cookies(services.public_prefix()); - let keys_an_identity = key_cookie_names.iter().any(|name| identity.contains(name)); + let keys_an_identity = key_cookie_names.iter().any(|name| { + crate::cookies::template_cache_policy::is_identity_cookie(services.public_prefix(), name) + }); let (cookie_disqualifies, cookie_values) = match evaluate_cookie_policy( req.headers(), key_cookie_names, @@ -5782,11 +5781,10 @@ pub async fn handle_publisher_request( let request_method = req.method().clone(); let mut origin_request = Some(req); - let eid_cookie_may_need_persistence = cookie_jar.as_ref().is_some_and(|jar| { - jar.get(&services.public_prefix().cookie_name(COOKIE_EIDS)) - .is_some() - || jar.get(COOKIE_SHAREDID).is_some() - }); + let eids_may_need_persistence = !ec_context.module_eids().is_empty() + || cookie_jar + .as_ref() + .is_some_and(|jar| jar.get(COOKIE_SHAREDID).is_some()); // This decision also gates the concurrent origin send below. A marker skip // cannot currently delay the origin behind an auction because marker // validation requires a non-empty registry, and every such auction sets @@ -5801,7 +5799,7 @@ pub async fn handle_publisher_request( && auction .registry .is_some_and(|registry| !registry.is_empty()), - eid_cookie_may_need_persistence, + eids_may_need_persistence, privacy_needs_row: ec_context.storage_withdrawn(), }); let mut pending_origin = None; @@ -5907,7 +5905,6 @@ pub async fn handle_publisher_request( apply_auction_eids_and_device( &mut auction_request, &AuctionEidTargeting { - cookie_jar: cookie_jar.as_ref(), ec_id, kv_snapshot: ec_context.kv_snapshot(), partner_registry: auction.registry, @@ -6711,7 +6708,6 @@ pub(crate) struct MatchedSlotsContext<'a> { /// Borrowed inputs for [`apply_auction_eids_and_device`], bundled to keep the /// helper within the project's 7-argument cap. struct AuctionEidTargeting<'a> { - cookie_jar: Option<&'a CookieJar>, ec_id: Option<&'a str>, kv_snapshot: &'a crate::ec::EcKvSnapshot, partner_registry: Option<&'a PartnerRegistry>, @@ -6732,12 +6728,8 @@ fn apply_auction_eids_and_device( auction_request: &mut AuctionRequest, targeting: &AuctionEidTargeting<'_>, ) { - let ts_eids_value = targeting - .cookie_jar - .and_then(|j| j.get(&targeting.services.public_prefix().cookie_name(COOKIE_EIDS))) - .map(|c| c.value().to_owned()); let client_eids = if targeting.ec_id.is_some() { - resolve_client_auction_eids(None, ts_eids_value.as_deref()) + resolve_client_auction_eids(None, targeting.ec_context.module_eids()) } else { None }; @@ -8347,7 +8339,6 @@ pub async fn handle_page_bids( .map(str::to_owned); let consent_context = ec_context.consent().clone(); let geo = ec_context.geo_info().cloned(); - let cookie_jar = handle_request_cookies(&req)?; // Same fail-closed jurisdiction-aware gate the publisher navigation path // uses — relies on the adapter's geo-aware EC context. @@ -8427,9 +8418,9 @@ pub async fn handle_page_bids( _ => crate::ec::EcKvSnapshot::NotRead, }; // Hand the loaded row to the request context so response - // finalization — which runs on an EC context the adapter owns, - // after this handler returns — ingests `ts-eids`/`sharedId` updates - // from this read instead of paying for a second lookup. + // finalization, which runs on an EC context the adapter owns + // after this handler returns, ingests the request's identifier + // updates from this read instead of paying for a second lookup. if !matches!(page_bids_kv_snapshot, crate::ec::EcKvSnapshot::NotRead) { ec_context.set_kv_snapshot(page_bids_kv_snapshot.clone()); } @@ -8444,7 +8435,6 @@ pub async fn handle_page_bids( apply_auction_eids_and_device( &mut auction_request, &AuctionEidTargeting { - cookie_jar: cookie_jar.as_ref(), ec_id: ec_id.as_deref(), kv_snapshot: &page_bids_kv_snapshot, partner_registry: auction.registry, @@ -9285,7 +9275,7 @@ mod tests { has_kv: true, marker_valid: false, auction_needs_row: false, - eid_cookie_may_need_persistence: false, + eids_may_need_persistence: false, privacy_needs_row: false, }; assert!(should_preload_ec_snapshot(&baseline)); @@ -9316,7 +9306,7 @@ mod tests { has_kv: true, marker_valid: true, auction_needs_row: false, - eid_cookie_may_need_persistence: false, + eids_may_need_persistence: false, privacy_needs_row: false, }; assert!(!should_preload_ec_snapshot(&marker_only)); @@ -9325,7 +9315,7 @@ mod tests { ..marker_only })); assert!(should_preload_ec_snapshot(&EcSnapshotPreloadInput { - eid_cookie_may_need_persistence: true, + eids_may_need_persistence: true, ..marker_only })); assert!(should_preload_ec_snapshot(&EcSnapshotPreloadInput { @@ -9846,11 +9836,16 @@ mod tests { .then(scheduling_slot) .into_iter() .collect::>(); - let mut request = navigation_request(); + let request = navigation_request(); if has_eid_cookie { - request - .headers_mut() - .insert(header::COOKIE, HeaderValue::from_static("ts-eids=present")); + ec_context.set_module_eids_for_test(vec![crate::openrtb::Eid { + source: "ssp.example.com".to_owned(), + uids: vec![crate::openrtb::Uid { + id: "present".to_owned(), + atype: Some(1), + ext: None, + }], + }]); } let integration_registry = test_registry(&settings); @@ -17294,7 +17289,6 @@ mod tests { Some(&graph), &partners, None, - None, &mut response, &services, ) @@ -32803,7 +32797,6 @@ mod tests { Some(&graph), &PartnerRegistry::empty(), None, - None, &mut response, &crate::platform::test_support::noop_services(), ) diff --git a/crates/trusted-server-core/src/test_support.rs b/crates/trusted-server-core/src/test_support.rs index b3c899ebda..e37d3798fe 100644 --- a/crates/trusted-server-core/src/test_support.rs +++ b/crates/trusted-server-core/src/test_support.rs @@ -352,19 +352,29 @@ pub(crate) mod logs { use std::sync::Once; thread_local! { - static CAPTURED: RefCell>> = const { RefCell::new(None) }; + /// The least severe level this thread keeps, and the lines kept. + static CAPTURED: RefCell)>> = + const { RefCell::new(None) }; } struct ThreadLogger; impl log::Log for ThreadLogger { - fn enabled(&self, _metadata: &log::Metadata<'_>) -> bool { - CAPTURED.with(|captured| captured.borrow().is_some()) + fn enabled(&self, metadata: &log::Metadata<'_>) -> bool { + CAPTURED.with(|captured| { + captured + .borrow() + .as_ref() + .is_some_and(|(level, _)| metadata.level() <= *level) + }) } fn log(&self, record: &log::Record<'_>) { + if !self.enabled(record.metadata()) { + return; + } CAPTURED.with(|captured| { - if let Some(lines) = captured.borrow_mut().as_mut() { + if let Some((_, lines)) = captured.borrow_mut().as_mut() { lines.push(format!("{} {}", record.level(), record.args())); } }); @@ -380,13 +390,27 @@ pub(crate) mod logs { /// warning level or above, as the level and then the message, such as /// `WARN ...`. A future that runs on another thread logs nothing here. pub(crate) async fn logged_by(future: F) -> (F::Output, Vec) { + logged_at(log::Level::Warn, future).await + } + + /// As [`logged_by`], keeping each line at `level` or above. + pub(crate) async fn logged_at( + level: log::Level, + future: F, + ) -> (F::Output, Vec) { INSTALL.call_once(|| { log::set_logger(&LOGGER).expect("should be the only logger the tests install"); - log::set_max_level(log::LevelFilter::Warn); + log::set_max_level(log::LevelFilter::Trace); }); - CAPTURED.with(|captured| *captured.borrow_mut() = Some(Vec::new())); + CAPTURED.with(|captured| *captured.borrow_mut() = Some((level, Vec::new()))); let output = future.await; - let lines = CAPTURED.with(|captured| captured.borrow_mut().take().unwrap_or_default()); + let lines = CAPTURED.with(|captured| { + captured + .borrow_mut() + .take() + .map(|(_, lines)| lines) + .unwrap_or_default() + }); (output, lines) } } diff --git a/docs/guide/api-reference.md b/docs/guide/api-reference.md index a26d5c70f3..b4ab8e3c61 100644 --- a/docs/guide/api-reference.md +++ b/docs/guide/api-reference.md @@ -516,7 +516,7 @@ curl 'https://edge.example.com/_ts/page-bids?path=%2Fnews&format=json' \ ## Edge Cookie Endpoints -Partners are configured statically in `[[ec.partners]]` and loaded into an in-memory registry at startup. There is no runtime partner-registration endpoint and the legacy browser pixel sync endpoint has been removed; browser-resolved IDs are ingested through Prebid EID cookies. +Partners are configured statically in `[[ec.partners]]` and loaded into an in-memory registry at startup. There is no runtime partner-registration endpoint and the legacy browser pixel sync endpoint has been removed; browser-resolved IDs are ingested through the identifier modules a build links, such as the Prebid module reading its `ts-eids` cookie. --- diff --git a/docs/guide/configuration.md b/docs/guide/configuration.md index 25369c6f5f..a9c8f7e543 100644 --- a/docs/guide/configuration.md +++ b/docs/guide/configuration.md @@ -3443,8 +3443,9 @@ The optional cookie lists control both template lookup and storage: Cookie names match exactly and case-sensitively: `session` and `Session` are different names. Use nonempty ASCII HTTP token names; whitespace, `;`, `=`, and non-ASCII characters are invalid. Configuration rejects invalid names, duplicates -within a list, overlap between lists, and TS identity cookies (`ts-ec`, `ts-eids`, -`sharedId`) in the key list. Identity cookies may be listed for bypass. Wildcards, +within a list, overlap between lists, and identity cookies in the key list, being +any cookie under the public prefix (such as `ts-ec`) and `sharedId`. Identity +cookies may be listed for bypass. Wildcards, prefixes, and regular expressions are not supported. With either list nonempty, parsing of all `Cookie` fields after existing request preparation bypasses duplicate key-cookie names, invalid names, and malformed key-cookie values. When independence is `true`, diff --git a/docs/guide/edge-cookies.md b/docs/guide/edge-cookies.md index 56b69e0169..c8f5b49a12 100644 --- a/docs/guide/edge-cookies.md +++ b/docs/guide/edge-cookies.md @@ -56,11 +56,11 @@ sequenceDiagram Note over TS: Phase 2: Routing (organic only)
generate_if_needed() TS->>TS: HMAC-SHA256(IP) + random suffix TS->>KV: Create entry (consent, geo, device) - Note over TS: Phase 3: Finalize
Ingest Prebid EID cookies + Note over TS: Phase 3: Finalize
Ingest the request's identifiers TS-->>B: Response + Set-Cookie: ts-ec=... else Return Visit (EC cookie present) Note over TS: Phase 2: Routing
EC exists, skip generation - Note over TS: Phase 3: Finalize
Ingest Prebid EID cookies + Note over TS: Phase 3: Finalize
Ingest the request's identifiers TS-->>B: Response
(no cookie refresh) end @@ -82,8 +82,8 @@ flowchart TD ExplicitWithdrawal -- "No" --> HeaderOnly ConsentCheck -- "Yes" --> WasPresent{EC was present
in request?} - WasPresent -- "Yes, not generated" --> Returning["Ingest Prebid EID cookies
No cookie or KV TTL refresh"] - WasPresent -- "No, just generated" --> NewEc["Ingest Prebid EID cookies
Set ts-ec cookie"] + WasPresent -- "Yes, not generated" --> Returning["Ingest the request's identifiers
No cookie or KV TTL refresh"] + WasPresent -- "No, just generated" --> NewEc["Ingest the request's identifiers
Set ts-ec cookie"] ``` When the required permissions are not set for the current request (for example a location lookup that failed, or a baseline that requires a signal no module has answered), Trusted Server fails closed for EC use by stripping EC headers, but it does **not** treat that as authoritative revocation of an already-issued EC. @@ -97,7 +97,7 @@ The Edge Cookie code never reads consent. It checks only whether the required ** - **A country and region baseline.** The country, and an optional region such as a US state, that the geo module returns. A region rule takes precedence over its country, and when no country is identified, or neither the region nor the country has a rule, the baseline on the top node of the rules tree applies. - **Consent and privacy signals.** TCF, GPP, a US Privacy string, GPC and the Model Terms for Marketing preference (`euconsent-v2`, `__gpp` / `__gpp_sid`, `us_privacy`, `Sec-GPC`, `__mtm_pref`), each read by its own [permission signal module](/guide/permission-signals) and mapped onto permissions as a **grant or a revoke** on top of that baseline, in the configured order. There is no separate consent KV fallback. -Every permission is resolved this way, and the Edge Cookie is created only when each permission its module requires is set. A location lookup that fails resolves at the requires-signal floor, so the cookie is not created unless a signal grants the permission. The core encodes no jurisdiction's law. The deployer brings the policy, and the per-country and per-region rules live in the policy file rather than in core logic. See the [permission model](/guide/permission-model) for the full list of permission sources and the resolution order. Writing browser-supplied Prebid EIDs into the identity graph, and server-side pull sync, additionally need the storage and personalised-ad permissions both set, the pair that also gates EIDs in the bid request. With storage alone the Edge Cookie stays first-party and partner sharing is withheld, so the EC ID is not sent to pull-sync partners. See [Prebid EID Flow](#prebid-eid-flow). +Every permission is resolved this way, and the Edge Cookie is created only when each permission its module requires is set. A location lookup that fails resolves at the requires-signal floor, so the cookie is not created unless a signal grants the permission. The core encodes no jurisdiction's law. The deployer brings the policy, and the per-country and per-region rules live in the policy file rather than in core logic. See the [permission model](/guide/permission-model) for the full list of permission sources and the resolution order. Writing the identifiers a request carries into the identity graph, and server-side pull sync, additionally need the storage and personalised-ad permissions both set, the pair that also gates EIDs in the bid request. With storage alone the Edge Cookie stays first-party and partner sharing is withheld, so the EC ID is not sent to pull-sync partners. See [Identifier Flow](#identifier-flow). ```mermaid flowchart TD @@ -159,7 +159,7 @@ Partner identities flow into the KV identity graph through three channels. Each ```mermaid flowchart LR subgraph Browser-initiated - Prebid["Prebid EIDs
/auction body + ts-eids + sharedId
Response-time ingestion"] + Prebid["Request identifiers
/auction body + identifier modules + sharedId
Response-time ingestion"] end subgraph Server-initiated @@ -172,17 +172,17 @@ flowchart LR Pull --> KV ``` -### Prebid EID Flow +### Identifier Flow -Browser-resolved Prebid EIDs reach the identity graph when Trusted Server finalizes a response. It collects matched partner UIDs from three request-local sources, applied in this order so that, when two sources carry the same partner, the later source's UID is the candidate offered to KV: +The identifiers a request carries reach the identity graph when Trusted Server finalizes a response. Core asks each identifier module the build links what it learned for the request, once, when it reads the Edge Cookie state, and collects matched partner UIDs from three request-local sources, applied in this order so that, when two sources carry the same partner, the later source's UID is the candidate offered to KV: -1. the `ts-eids` cookie, +1. what the identifier modules learned, such as the Prebid module's `ts-eids` cookie, 2. the `eids` array in the `/auction` request body, 3. the `sharedId` cookie. -The `/auction` body carries the full EID set Prebid.js resolved, so `/auction` ingestion is not limited by the size cap TSJS applies to the `ts-eids` cookie. The cookie still bridges Prebid user ID modules with the identity graph on requests that carry no EID body, such as `GET /_ts/page-bids` and page navigations. +The `/auction` body carries the full EID set Prebid.js resolved, so `/auction` ingestion is not limited by the size cap the Prebid page script applies to its cookie. The cookie still bridges Prebid user ID modules with the identity graph on requests that carry no EID body, such as `GET /_ts/page-bids` and page navigations. -These Prebid EID writes follow the same permission rule as bidstream EID forwarding: the storage and personalised-ad permissions must both be set. A reader who allows storage but not personalised ads keeps their EC, but no EIDs from the request are persisted. +These writes follow the same permission rule as bidstream EID forwarding: the storage and personalised-ad permissions must both be set. A reader who allows storage but not personalised ads keeps their EC, but no EIDs from the request are persisted. ```mermaid sequenceDiagram @@ -190,6 +190,7 @@ sequenceDiagram participant TSJS as TSJS Prebid Module participant B as Browser Cookie Jar participant TS as Trusted Server + participant PM as Prebid Module participant KV as KV Store Prebid->>Prebid: Auction completes @@ -201,15 +202,18 @@ sequenceDiagram Note over B,TS: Next eligible EID-sync request B->>TS: Document navigation, POST /auction,
or admitted GET /_ts/page-bids - TS->>TS: Base64 decode → parse OpenRTB-style EIDs
match source domains to partners + TS->>PM: Identifiers for this request? + PM->>PM: Read ts-eids, base64 decode,
parse OpenRTB-style EIDs + PM-->>TS: [{source, uids}] + TS->>TS: Match source domains to partners TS->>KV: Add missing partner IDs in one conditional write
skip when UIDs already match
(requires the storage and personalised-ad permissions) ``` -Current TSJS writers preserve the full OpenRTB-style `{source, uids:[...]}` shape in `ts-eids`. The server remains backward-compatible with earlier flattened `{source, id, atype}` cookies during rollout, but new cookies use the structured `uids[]` form. +Core names no cookie here. The Prebid module, `crates/auction/prebid`, owns `ts-eids`, its two shapes and its size limit, and hands core what the cookie holds through the identifier module interface, `trusted_server_core::ec::identifiers::IdentifierModule`. The module reads the structured `{source, uids:[...]}` shape the script writes today and the earlier flattened `{source, id, atype}` shape, so a cookie written before the script changed is still read. Another module that learns identifiers for a request, from a cookie or a store of its own, offers itself the same way, and core syncs what it learns to the identity graph and carries it to the auction with no change to core. The `sharedId` cookie follows a similar path but is written directly by Prebid's SharedID module rather than by TSJS. The server reads it separately and maps it via the `sharedid.org` source domain. -Returning-user cookie persistence runs only on publisher document navigations, `POST /auction`, and admitted `GET /_ts/page-bids` SPA navigations. Static assets, analytics, integration requests, filter short circuits, and other subresources do not decode or persist these cookies. New EC creation remains eligible regardless of route because its backing row must include the request's initial IDs. +Returning-user persistence runs only on publisher document navigations, `POST /auction`, and admitted `GET /_ts/page-bids` SPA navigations. Static assets, analytics, integration requests, filter short circuits, and other subresources do not persist what the request carries. Core still asks the identifier modules on these requests wherever it reads the Edge Cookie state with an Edge Cookie module selected, so the Prebid module decodes its cookie on them too. New EC creation remains eligible regardless of route because its backing row must include the request's initial IDs. Each eligible request attempts at most one conditional cookie update. If another writer wins, Trusted Server reads the row once and does not write again from that request. A matching value completes the sync; an absent or different value is deferred. @@ -251,7 +255,7 @@ sequenceDiagram DSP-->>PS: OpenRTB bid response PS-->>TS: OpenRTB seatbid response TS-->>B: Auction response + x-ts-eids header when available - TS->>KV: Add missing partner UIDs from request eids[]
+ ts-eids/sharedId cookies (storage and personalised-ad permissions) + TS->>KV: Add missing partner UIDs from request eids[]
+ what the Prebid module read from ts-eids and the sharedId cookie
(storage and personalised-ad permissions) ``` The relevant OpenRTB structure forwarded to Prebid Server and downstream partners is: @@ -289,7 +293,7 @@ The relevant OpenRTB structure forwarded to Prebid Server and downstream partner } ``` -Server-resolved EIDs and current-request Prebid EIDs are deduplicated by `source + uid.id`. Browser-supplied EIDs, from the `/auction` request body, the `ts-eids` cookie, or the `sharedId` cookie, only fill partner UIDs that are missing from KV; a different stored UID is kept, as described in [Prebid EID Flow](#prebid-eid-flow). +Server-resolved EIDs and current-request Prebid EIDs are deduplicated by `source + uid.id`. Browser-supplied EIDs, from the `/auction` request body, an identifier module such as the Prebid module reading its `ts-eids` cookie, or the `sharedId` cookie, only fill partner UIDs that are missing from KV; a different stored UID is kept, as described in [Identifier Flow](#identifier-flow). ### Pull-Sync Completeness Marker diff --git a/docs/guide/integration-guide.md b/docs/guide/integration-guide.md index 416d8033ff..cce976ce04 100644 --- a/docs/guide/integration-guide.md +++ b/docs/guide/integration-guide.md @@ -468,6 +468,7 @@ builder every integration uses. | ----------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- | | `.with_proxy(...)` | Routes the paths the proxy declares | | `.with_request_filter(...)` | Inspects a request and can turn it back before it reaches the origin | +| `.with_identifier_module(...)` | Hands core the identifiers the module learned, asked at each Edge Cookie read when `[ec] module` names a module | | `.with_middleware(...)` | Offers a page change that an entry may place by the middleware's name | | `.with_bundle_tag_attribute(name, value)` | Puts an attribute on the script bundle's tag, for a browser module that reads a setting from it | | `.with_js_module(CarriedJsModule { source, sha256 })` | Carries the integration's own browser script, built outside `trusted-server-js` | @@ -884,6 +885,26 @@ than holding a copy. A public prefix's parts are letters, digits, `_`, `-` and `.`, and the first starts with a letter or a single `_`, so every form is a valid cookie name and no cookie name starts with `__`. +### Handing core the identifiers a request carries + +A crate whose page script persists identifiers in the browser, or that learns +them for a request another way, hands them to core by declaring an +`IdentifierModule` with `.with_identifier_module(...)`. No section selects +one. Core asks every running registration's module once per request, when it +reads the Edge Cookie state and `[ec] module` names a module, through a module +call carrying the request's evidence, settings, services, consent, client +information and resolved permissions, and its geo when the location was found, +and the module names what it reads with `call.inject`. What it returns is a +list of OpenRTB-style `Eid` entries, in the order it offers them, so that +where two carry the same partner the later one is the candidate. + +Core syncs what every module learned to the identity graph when it finalizes +the response, after the request body's own EIDs, and forwards it in the +auction when the body carries none. The cookie a module reads is the module's +own, named through `services.public_prefix().cookie_name(...)`, and core +names it nowhere. The Prebid module, `crates/auction/prebid`, is the module +that reads the `ts-eids` cookie its script writes. + ### A vendor's shared table What every module of one vendor needs, such as an endpoint and a license key, diff --git a/docs/guide/integrations/prebid.md b/docs/guide/integrations/prebid.md index 329079aac5..7bdd742df3 100644 --- a/docs/guide/integrations/prebid.md +++ b/docs/guide/integrations/prebid.md @@ -861,7 +861,8 @@ page or auction. When available, the opaque value follows the standard path: an EC/KV partner configured with `source_domain = "liveramp.com"`, when that partner UID is not already stored. 5. The browser also persists the opaque value in the bounded `ts-eids` cookie, - which lets requests without an EID body, such as `GET /_ts/page-bids` and + which this module reads and hands to core as the identifiers the request + carries, so requests without an EID body, such as `GET /_ts/page-bids` and page navigations, ingest it. Trusted Server treats the RampID envelope as an opaque string. Do not log, @@ -950,9 +951,9 @@ Trusted Server uses a **hybrid EID forwarding model** for Prebid-routed auctions 2. **Server-side EIDs from the EC/KV identity graph** are resolved on the edge from the current EC ID. 3. Trusted Server **merges and deduplicates** both sets before calling Prebid Server. 4. The merged result is forwarded downstream as `user.ext.eids` in the OpenRTB request. -5. After the response, Trusted Server adds matched partner UIDs that are missing from the EC identity graph, reading the `ts-eids` cookie, then the `/auction` request-body EIDs, then the `sharedId` cookie. Because the body carries the untrimmed EID set, `/auction` ingestion is not limited by the size-capped `ts-eids` cookie. +5. After the response, Trusted Server adds matched partner UIDs that are missing from the EC identity graph, reading what this module learned from the `ts-eids` cookie, then the `/auction` request-body EIDs, then the `sharedId` cookie. Because the body carries the untrimmed EID set, `/auction` ingestion is not limited by the size-capped `ts-eids` cookie. -The `ts-eids` cookie still matters for requests without an EID body, such as `GET /_ts/page-bids` and page navigations, where it is the source for both EID fallback and identity-graph ingestion. Identity-graph EID writes follow the same permission rule as bidstream forwarding: the storage and personalised-ad permissions must both be set. +The `ts-eids` cookie still matters for requests without an EID body, such as `GET /_ts/page-bids` and page navigations, where it is the source for both EID fallback and identity-graph ingestion. This module owns the cookie, its two shapes and its size limit, and hands core what it holds through the identifier module interface, `trusted_server_core::ec::identifiers::IdentifierModule`, which core asks once per request when it reads the Edge Cookie state. Core names the cookie nowhere. Identity-graph EID writes follow the same permission rule as bidstream forwarding: the storage and personalised-ad permissions must both be set. This means Prebid auctions get same-request transparency for browser-resolved IDs without giving up the durability of the server-managed EC identity graph. diff --git a/docs/superpowers/specs/2026-10-14-identifier-modules-design.md b/docs/superpowers/specs/2026-10-14-identifier-modules-design.md new file mode 100644 index 0000000000..38e6dd1ca8 --- /dev/null +++ b/docs/superpowers/specs/2026-10-14-identifier-modules-design.md @@ -0,0 +1,106 @@ +# Design Spec: A module hands core the identifiers it learned for a request + +**Status:** Proposed, 2026-10-14. Raised as draft pull requests jwrosewell/trusted-server#36 and IABTechLab/trusted-server#1382, for Tech Lab review. +**Author:** 51Degrees (contributed), for Tech Lab review +**Related specs:** `2026-07-10-kv-eid-request-snapshot-ec-recovery-design.md`, `2026-09-08-1138-per-cookie-template-cache-policy-design.md` +**Related PRs:** jwrosewell/trusted-server#36 (fork draft) and IABTechLab/trusted-server#1382 (Tech Lab draft). IABTechLab/trusted-server#1342 holds every design document of the stack. +**Issues:** IABTechLab/trusted-server#1337 +**Last updated:** 2026-10-15 + +> **Why this spec exists.** Core parses the `ts-eids` cookie the Prebid page script writes, in both the shape it writes today and the one it wrote before, and syncs what it finds into the identity graph. Core accepts up to 8 KiB of the cookie's value where the script trims what it writes to 3072 bytes. So core knows one vendor's cookie, and the Prebid crate does not own the cookie its own script writes. + +## 1. Problem + +`crates/trusted-server-core/src/ec/prebid_eids.rs` holds the cookie's two JSON shapes, its raw size limit of 8 KiB and the parser. The cookie's name, `eids` under the public prefix, is a separate constant, `COOKIE_EIDS` in `crates/trusted-server-core/src/constants.rs`, which the Fastly adapter, the auction endpoints, the publisher path and the template cache's identity rule each use. The auction endpoints call the parser to fall back to the cookie when a request body carries no EIDs, the publisher path calls it for a navigation's and a page-bids request's auction, and the Fastly adapter extracts the cookie's raw value before routing so that response finalization can parse it afterwards, when the request is gone. + +A second script that persisted identifiers in a cookie of its own, or a module that learned them from a store, would have no way to hand them to core short of a second parser in core. + +## 2. Design + +**Core declares the interface.** `IdentifierModule`, in `trusted_server_core::ec::identifiers`, is a module that learns identifiers for a request. It is called with a module call, as every module is, carrying the request's evidence, settings, services, consent, client information and resolved permissions, and its geo when the location was found, and it names what it reads with `call.inject`. The call carries neither the Edge Cookie identifier nor the device signals, because core puts them in a module's context only after the state is built, so a module that names either is withheld. The module returns OpenRTB-style `Eid` entries, in the order it offers them, so that where two carry the same partner the later one is the candidate. A call that withholds a value the module names contributes nothing and is logged, so one module's fault does not lose another's identifiers. + +**A registration offers one.** `IntegrationRegistrationBuilder::with_identifier_module` declares it. No section selects one. The registry collects the module of every running registration, in registration order, and claims each module's name, so two registrations cannot declare the same one. Each adapter hands that list to the services it builds for a request with `RuntimeServices::with_identifier_modules`, as it hands them the registry's geo, Edge Cookie and device modules. The permission signal modules are different, because each adapter builds their list from the settings and gives it to the services builder. + +**Core asks once, when it reads the Edge Cookie state.** That is where the request's evidence is in hand and where the permission signal modules are already asked. The answers are held on the `EcContext` as `module_eids`, and from there they reach response finalization, which offers them to the identity graph, and the auction, which forwards them when the request body carries none. With no Edge Cookie module selected nothing can hold or forward an identifier, so none are asked. + +**Permissions apply to uses and not to asking.** Core asks regardless of what the request's permissions resolve to, and the call carries the resolved permissions so that a gated value is refused to a module that does not declare what the value needs. `PrebidEids` declares no permission and names only the evidence and the services, which are never gated, so it decodes the cookie of a reader whose permissions forbid every use of what it finds. Every use stays behind a gate. Finalization returns before it ingests anything when the Edge Cookie is not permitted, and it drops the updates unless both `StoreOnDevice` and `SelectPersonalisedAds` are set. The auction and the publisher path forward module entries only for a reader whose Edge Cookie may be shared, which needs the same two permissions, and then pass the merged entries through the bid request's permission gate. + +**Where each adapter asks.** All four adapters call `RuntimeServices::with_identifier_modules` when they build the services for a request, and each asks the modules wherever it reads the Edge Cookie state. An adapter that left the call out would give its services an empty list, and every request would be answered as if no module were linked, with no error. No test checks the four calls. Only the Fastly adapter finalizes a response with `ec_finalize_response`, so only on Fastly are the answers synced into the identity graph. The other three adapters pass no identity graph to their handlers, so on them the answers feed only the EIDs forwarded in an auction. + +| Adapter | Hands the list to a request's services in | Reads the Edge Cookie state, and so asks, for | Finalizes with `ec_finalize_response` | +| ---------- | ----------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | +| Fastly | `build_per_request_services` in `crates/trusted-server-adapter-fastly/src/app.rs` | Every named route except configuration, batch sync and data, and the fallback | Yes, from `apply_edgezero_ec_finalize` in `crates/trusted-server-adapter-fastly/src/lib.rs` | +| Axum | `AppState::services_for_request` in `crates/trusted-server-adapter-axum/src/app.rs` | `/auction`, `/_ts/page-bids`, the permissions page, module routes and the publisher fallback | No | +| Cloudflare | `AppState::services_for_request` in `crates/trusted-server-adapter-cloudflare/src/app.rs` | `/auction`, `/_ts/page-bids`, the permissions page, module routes and the publisher fallback | No | +| Spin | `AppState::services_for_request` in `crates/trusted-server-adapter-spin/src/app.rs` | `/auction`, `/_ts/page-bids`, the permissions page, module routes and the publisher fallback | No | + +The fallback is where each adapter serves the publisher's pages and subresources, and a subresource asks the modules as a page does. Two kinds of request are answered before the read. A closed path (`/_ts/admin` and `/admin/keys`) is answered `404` on every adapter, and a `GET` under `/static/tsjs=`, which serves the script bundles, is answered first on Axum, Cloudflare and Spin and asks the modules on Fastly. + +**What it costs.** The Prebid module reads its cookie and, when the cookie is present, decodes it whenever core asks, which is on every read of the Edge Cookie state once an Edge Cookie module is selected. Before this change the cookie was decoded only where its answer could be used, being the auctions, the finalization of a request marked for identity sync and the finalization that creates an Edge Cookie, and both finalizations decoded it only when a partner registry was configured. Core reads the state before the handler decides whether a request is eligible, so asking at the read reaches requests that never use an answer, such as the publisher's images, scripts and style sheets on every adapter. The asking also happens when no identity graph or partner is configured. The work is a scan of the `Cookie` header and, when the cookie is present, a base64 decode of at most 8 KiB and a JSON parse that tries the legacy shape and then the shape the script writes today, once for each request, because the answer is held on the request state. The call is synchronous, so a module that read a store would make that read inside the request, on the same requests. The interface gives a module no sign of whether its answer will be used. What reaches the identity graph is unchanged, because finalization still ingests only for a request marked for identity sync or one that creates an Edge Cookie. `2026-07-10-kv-eid-request-snapshot-ec-recovery-design.md` says that other subresources neither decode nor persist the cookies, and the guide said the same until this change. The earlier design's sentence is now false for decoding. The guide (`docs/guide/edge-cookies.md`) now says that the modules are asked on other subresources too, and the earlier design is not amended. + +**The order of the sources does not change.** Finalization collects partner updates from what the modules learned, then from the request body's own EIDs, which win on conflicts because a module's cookie is size-capped and may be missing partners the body still carries in full, then from the `sharedId` cookie. The auction forwards the body's EIDs, or what the modules learned when the body carries none, gated on the Edge Cookie as before. Core adds no rule for two modules that offer one source. Both entries are kept in registration order, and finalization takes the later one as the candidate for a partner. The auction copies the entries as they are and then merges them by source, which keeps each identifier once, fills a missing `atype` or `ext` from the later entry and drops blank identifiers and identifiers longer than 512 bytes, so two modules that offer one source forward one entry for it. + +**The Prebid crate owns its cookie.** `crates/auction/prebid/src/eids.rs` carries the cookie's name, its two shapes, its size limit and the parser, and `PrebidEids` is the first identifier module, registered from the Prebid builder. It reads the cookie through the request's evidence, named through the public prefix the services carry, so core names the cookie nowhere, as it names the Model Terms cookie nowhere. The script that writes the cookie is `syncPrebidEidsCookie` in `crates/trusted-server-js/lib/src/integrations/prebid/index.ts`. It names the cookie after the page's public prefix, writes the structured shape and trims the value to 3072 bytes, where the module accepts 8 KiB and both shapes. No shared definition and no test ties the two sides together. The script's tests (`crates/trusted-server-js/lib/test/integrations/prebid/index.test.ts`) check what it writes, and the module's tests check what it reads from values written by hand, so a change on one side that the other does not follow fails neither suite. The interface does not make a module name its cookie through `PublicPrefix::cookie_name`, and the template cache's identity rule below depends on that convention. + +**What leaves core.** `COOKIE_EIDS`, the parser and its tests, the cookie's extraction in the Fastly adapter, and the `eids_cookie` parameter of `ec_finalize_response`. Three of these are public, so a crate outside core that used one breaks, being `trusted_server_core::constants::COOKIE_EIDS`, the module `trusted_server_core::ec::prebid_eids` with its `parse_prebid_eids_cookie`, and the `eids_cookie` parameter. `IntegrationRegistration` also gains the public field `identifier_module`, so a crate that builds one with a struct literal and not through its builder breaks as well. The publisher path also changes what makes it read the identity-graph row ahead of the origin response. It used to read the row when a `ts-eids` or a `sharedId` cookie was present, and now reads it when the modules learned an identifier or a `sharedId` cookie is present, so a `ts-eids` cookie that cannot be parsed, or that holds nothing usable, no longer triggers the read. + +**The template cache's identity rule.** `template_cache_key_cookies` may not name an identity cookie, because a shared template keyed on a cookie that differs for each reader would belong to one reader. The rule refused three names, `ts-ec`, `ts-eids` and `sharedId`, compared exactly. It now refuses `sharedId` and every name under the public prefix, and it compares the prefix without regard to case. The effects are: + +- At load the check uses core's own prefix, `ts`. A settings document whose `template_cache_key_cookies` lists `ts-ecr`, `ts-ec-pull-complete`, `ts-tester`, any other name that starts with `ts-`, or a name such as `TS-EC` that differs from a refused one only in case, loaded before and now fails to load with a configuration error naming the field and the cookie. The example settings file and the guide name only `ab_bucket`, and this repository cannot see which deployments list others. +- `template_cache_bypass_cookies` still accepts these names. +- Under a deployment's own prefix the check runs on each request and is not an error, so a key cookie under that prefix makes the request share no template. Before, only the Edge Cookie's name and the EIDs cookie's name did so, and now every name under that prefix does. +- The rule's reason, that every cookie under the prefix is one reader's own state, holds for the Edge Cookie. It does not hold for `ts-tester`, which holds `true` for every tester, or for `ts-ecr`, which holds `1` for every reader, so a deployment that keyed a shared template on `ts-tester`, to give testers a template of their own for example, can no longer do so. +- `docs/guide/configuration.md` states the new rule. The per-cookie template cache design (`2026-09-08-1138-per-cookie-template-cache-policy-design.md`, section 4.1) still lists `ts-ec`, `ts-eids` and `sharedId`, and this change does not amend it. + +**When a module fails or answers too much.** A call that withholds a value its module names contributes nothing and is logged at `warn` on every request where that happens, and the other modules are still asked. `PrebidEids` names only the evidence and the services, which every call carries, so a real request never withholds either from it. A cookie it cannot parse, being over 8 KiB, not base64, or not one of the two JSON shapes, contributes nothing and is logged at `trace` only, so a script that drifts from the reader shows as missing identifiers and not as a message at the default log level. Core applies no limit to what a module returns. The request body keeps at most 64 sources and 32 identifiers for each source and drops an entry whose source name is over 255 bytes. A module's entries are held to none of these limits, so the 8 KiB limit that the Prebid parser applies to its cookie is the only bound on what that module returns. Entries with no usable identifier are dropped later, by the auction's merge and by finalization, and finalization skips a source that no configured partner names. + +## 3. What is not in this change + +- The `sharedId` cookie, which Prebid's SharedID module writes directly. Core still reads it by name, because it is not the cookie this change is about and because folding it into the identifiers a module hands core would forward it in bid requests, where today it reaches the identity graph alone. The name is `COOKIE_SHAREDID` in `crates/trusted-server-core/src/constants.rs`, which the Fastly adapter and the publisher path read and the template cache's identity rule refuses. `crates/trusted-server-core/src/ec/identifiers.rs` takes the cookie's value and maps it to the partner whose source domain is `sharedid.org`. So core still names one cookie and one source domain that belong to Prebid's SharedID module. No follow-up to move them is raised yet, and the open questions ask for one. +- The OpenRTB `Eid` type the interface returns, which stays in core's `openrtb` module (`crates/trusted-server-core/src/openrtb.rs`), where the auction and the identity code both use it. +- **Settings.** This change adds no setting, and no section selects an identifier module. The one setting whose accepted values change is `template_cache_key_cookies`, described under the template cache's identity rule. + +## 4. Tests + +- Core asks each module in order and keeps their order, passes a module the evidence it names, and skips a module whose call withholds what it names while the next still answers. +- The registry hands out every running registration's module in registration order, hands out none for a registration no section selects, and refuses two registrations that declare one module name. +- Partner updates are collected from what the modules learned, the body's EIDs win over a module's stale value and ingest a partner the module's cookie trimmed, the `sharedId` cookie is applied last, an unknown source is skipped, an oversized identifier is skipped for a later valid one, and an empty registry touches no store. +- No line logged for a skipped EID carries the source the reader sent, and the line for a partner's EID names the partner's configured domain (`no_log_line_carries_the_source_a_reader_sent` in `ec/identifiers.rs`). +- The auction falls back to what the modules learned when the body carries none, prefers the body when it carries EIDs, and resolves nothing when neither has any. +- The Prebid module reads the cookie named by the public prefix, learns nothing from a request without it or with one it cannot read, is withheld without the evidence it names, parses both shapes, preserves a vendor-specific `atype`, skips entries with nothing to offer, and refuses invalid base64, invalid JSON and an oversized value. +- A template key cookie under the public prefix, and `sharedId`, are refused, and either may be listed for bypass. +- The publisher path still reads the identity-graph row ahead of the origin response when a valid completeness marker is present and the modules learned an identifier. + +Six joins have no test: + +1. No test reads a request through `EcContext::read_from_request*` with an identifier module in the services, so nothing checks that the answers reach `module_eids`, that the call carries the evidence, that a request with no Edge Cookie module selected is not asked, or that a request is asked once. The tests that sent a `ts-eids` cookie through the auction, the publisher path and finalization now set `module_eids` with `set_module_eids_for_test`, so the consumers are tested and the step that fills the field from a request is not. +2. No test checks that an adapter hands the registry's identifier modules to its services. The seam probe (`crates/testing/seam-probe`) declares a geo, Edge Cookie, device, prefix and robots.txt module, is exercised through the Axum and Fastly adapters, and declares no identifier module. +3. No test runs the Prebid module through the registry, from `with_identifier_module` in the Prebid builder to an answer for a request. +4. No test reads the cookie under a prefix other than `ts`, and the template rule on a request under a deployment's own prefix is tested for the Edge Cookie's name (`_ex-ec`) and no other name. +5. No test ties the script's cookie to the Prebid module's. +6. No test covers a module whose answer is larger than the request body's limits. + +## 5. Open questions + +These are for the reviewer. This revision records what the code does and decides none of them. + +1. Should core ask only on requests whose answer will be used? That is known only after the state is read, and response finalization no longer has the request. +2. Should the template cache's identity rule be narrowed so that it does not refuse the flags `ts-tester` and `ts-ecr`, or kept as it is with the deployments it breaks named? +3. Should a module's identity cookies reach that rule by requiring every module to name its cookies through `PublicPrefix::cookie_name`, or by letting a module declare them? Nothing requires the first today. +4. Should `RuntimeServicesBuilder::identifier_modules` be removed? No code calls it, because the adapters call `RuntimeServices::with_identifier_modules` on the services they have built. +5. Should a follow-up move the `sharedId` cookie and the `sharedid.org` source domain out of core? +6. Should a check tie the script's cookie to the Prebid module's, as `the_client_script_reads_the_attribute_the_server_writes` in `crates/trusted-server-core/src/prefix.rs` does for the public prefix attribute? + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/41-prebid-eids` at `e87090be0`. Issues found, four of them blocking. The blocking issues were that decoding moved onto every read of the Edge Cookie state without the spec saying so, that the adapters were unspecified and their wiring untested, that the template cache rule refused names that loaded before with no compatibility analysis, and that the opening sentence said core applies the script's size limit when core accepts 8 KiB and the script writes 3072 bytes. This revision states the per-request decoding and its cost, adds the adapter table and the joins with no test, lists what now fails to load, corrects the size limits, the locations of the cookie's name, `sharedId` and `Eid`, one sentence of the Edge Cookies guide and one cell of the integration guide, and puts the choices it does not make under open questions. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `d6ca6df80`, of the round 1 fixes and the new text. Approved, with no blocking issues. This revision corrects the account of how the cookie is parsed and how the request body treats a long source name, names the requests answered before the read, aligns one paragraph of the integration guide with the spec, and names the pull requests in the header. + +## Revision record + +| Date | Change | +| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-14 | First draft. | +| 2026-10-15 | Fixes from review round 1. The decoding on every read and its cost, the adapter table, the template cache rule's effects, the size limits, the locations of the cookie's name, `sharedId` and `Eid`, the permission order, failure and merge behavior, the untested joins, one sentence of the Edge Cookies guide, one cell of the integration guide, the open questions and the review log. | +| 2026-10-15 | Fixes from review round 2. How the cookie is parsed, how the request body treats a long source name, the requests answered before the read, one paragraph of the integration guide, and the pull requests in the header. | +| 2026-10-18 | No debug line for a skipped EID carries the source the reader sent, and the test for it. |