Unknown integration IDs and fields in [integrations.*] pass validation and are ignored #1201

Description

ts config validate, ts config push and runtime startup accept two kinds of mistakes under [integrations] with no error and no log line:

1. An unknown integration ID, such as [integrations.prebdi] for [integrations.prebid].
2. An unknown key inside the table of 10 of the 15 integrations, whether the block is enabled or disabled. For example enable = true instead of enabled = true.

The setting the operator meant never applies. The integration stays off or keeps its default, and nothing says why. ts config push stores the stray keys in the blob unchanged.

This contradicts the configuration guide, which says "Trusted Server rejects unknown TOML keys in runtime configuration" (docs/guide/configuration.md:179-183, added in #799). The root Settings struct and most section structs use #[serde(deny_unknown_fields)], and a comment in settings.rs:2598-2601 gives the reason: "an operator typo ... must fail config load loudly, not be silently ignored." A typo in a top-level section name gets a precise error that names the key. Integration tables are the main exception. Two top-level sections have the same gap: [tester_cookie] (TesterCookieConfig, settings.rs:2728) and [tinybird] (TinybirdSettings, settings.rs:1849). ts config validate accepts both [tester_cookie] enable = true and [tinybird] enabeld = true.

Two documented blocks already fall into this gap:

• docs/guide/integrations/lockr.md:43-49 shows organization_id and project_id under [integrations.lockr], and the table at :57-58 lists both as required. LockrConfig has neither field. The block as documented fails with a message that does not say why (it lacks the required app_id). Once app_id is added it validates, and both documented keys are ignored.
• Until Add full-surface documentation refresh spec #1049 (the parent of a4e01eb), docs/guide/integrations/kargo.md and gam.md showed [integrations.kargo] and [integrations.gam] blocks with enabled = true. No code has ever read either section. A config copied from those pages validates and does nothing.

The raw integration map dates from #113 (2025-11-20). deny_unknown_fields was later added to five integrations one PR at a time (DataDome in #769, Osano in #743, GPT diagnostics in #974, Prebid and APS in #1016). The other ten never got it.

Steps to reproduce

With the CLI

Save this as ts-typo.toml in the repository root:

[[handlers]]
path = "^/_ts/admin"
username = "admin"
password = "admin_password"

[publisher]
domain = "publisher.example.com"
cookie_domain = ".publisher.example.com"
origin_url = "https://origin.publisher.example.com"
proxy_secret = "publisher_proxy_secret"

[ec]
passphrase = "ec_passphrase"

# Typo of [integrations.prebid]
[integrations.prebdi]
enabled = true

# Typo of `enabled`; sourcepoint defaults to disabled
[integrations.sourcepoint]
enable = true

# testlight's key name; gpt reads `rewrite_script`
[integrations.gpt]
enabled = true
rewrite_scripts = false

Run:

cargo run -q -p trusted-server-cli --target "$(rustc -vV | sed -n 's/host: //p')" -- \
  config validate --app-config ts-typo.toml --no-env

Observed (exit 0):

[edgezero] config validate (typed): edgezero.toml + ts-typo.toml OK

For contrast, the same base config with [integrationz.prebid] (exit 2):

[ts] failed to deserialise ts-root-typo.toml into trusted_server_core::config::TrustedServerAppConfig: unknown field `integrationz`, expected one of `publisher`, `tester_cookie`, `trusted_client_ip`, `ec`, `integrations`, `handlers`, `response_headers`, `request_signing`, `rewrite`, `auction`, `consent`, `cache`, `proxy`, `creative_opportunities`, `image_optimizer`, `tinybird`, `debug`

And with [integrations.osano] plus made_up_key = "x", one of the five integrations that do reject unknown keys (exit 2). The message does not name the key:

[ts] typed app-config failed validation: trusted_server: Configuration error: Integration 'osano' configuration could not be parsed

As failing tests

Add to the mod tests block of crates/trusted-server-core/src/config.rs:

#[test]
fn deploy_validation_rejects_unknown_integration_id() {
    let mut settings = valid_settings();
    settings
        .integrations
        .insert("prebdi".to_owned(), serde_json::json!({ "enabled": true }));

    let err = validate_settings_for_deploy(&settings)
        .expect_err("should reject an unknown integration id");
    assert!(
        err.to_string().contains("prebdi"),
        "should name the unknown integration id: {err}"
    );
}

#[test]
fn deploy_validation_rejects_unknown_integration_fields() {
    let cases = [
        (
            "adserver_mock",
            serde_json::json!({ "endpoint": "https://mediator.example.com/mediate" }),
        ),
        ("didomi", serde_json::json!({})),
        (
            "google_tag_manager",
            serde_json::json!({ "container_id": "GTM-ABC1234" }),
        ),
        ("gpt", serde_json::json!({})),
        (
            "js_asset_proxy",
            serde_json::json!({ "assets": [{
                "path": "/assets/vendor.js",
                "origin_url": "https://cdn.example.com/vendor.js"
            }] }),
        ),
        ("lockr", serde_json::json!({ "app_id": "example-app" })),
        ("nextjs", serde_json::json!({})),
        (
            "permutive",
            serde_json::json!({ "organization_id": "example", "workspace_id": "example" }),
        ),
        ("sourcepoint", serde_json::json!({})),
        (
            "testlight",
            serde_json::json!({ "endpoint": "https://testlight.example.com/auction" }),
        ),
    ];
    let mut accepted = Vec::new();
    for (integration_id, base) in cases {
        for enabled in [true, false] {
            let mut config = base.clone();
            config["enabled"] = serde_json::json!(enabled);
            let mut settings = valid_settings();
            settings
                .integrations
                .insert(integration_id.to_owned(), config.clone());
            let baseline = validate_settings_for_deploy(&settings);
            assert!(
                baseline.is_ok(),
                "should accept the baseline {integration_id} config: {baseline:?}"
            );

            config["made_up_key"] = serde_json::json!("x");
            settings
                .integrations
                .insert(integration_id.to_owned(), config);
            match validate_settings_for_deploy(&settings) {
                Ok(()) => accepted.push(format!("{integration_id} (enabled = {enabled})")),
                Err(err) => assert!(
                    format!("{err:?}").contains("made_up_key"),
                    "should name the unknown field for {integration_id}: {err:?}"
                ),
            }
        }
    }

    assert!(
        accepted.is_empty(),
        "should reject an unknown field in every integration table, accepted: {accepted:?}"
    );
}

Run (the core test suite also runs natively on the host target):

cargo test -p trusted-server-core --lib --target "$(rustc -vV | sed -n 's/host: //p')" \
  -- config::tests::deploy_validation_rejects_unknown_integration

Observed:

test config::tests::deploy_validation_rejects_unknown_integration_id ... FAILED
test config::tests::deploy_validation_rejects_unknown_integration_fields ... FAILED

---- config::tests::deploy_validation_rejects_unknown_integration_id stdout ----
should reject an unknown integration id: ()

---- config::tests::deploy_validation_rejects_unknown_integration_fields stdout ----
should reject an unknown field in every integration table, accepted: ["adserver_mock (enabled = true)", "adserver_mock (enabled = false)", "didomi (enabled = true)", "didomi (enabled = false)", "google_tag_manager (enabled = true)", "google_tag_manager (enabled = false)", "gpt (enabled = true)", "gpt (enabled = false)", "js_asset_proxy (enabled = true)", "js_asset_proxy (enabled = false)", "lockr (enabled = true)", "lockr (enabled = false)", "nextjs (enabled = true)", "nextjs (enabled = false)", "permutive (enabled = true)", "permutive (enabled = false)", "sourcepoint (enabled = true)", "sourcepoint (enabled = false)", "testlight (enabled = true)", "testlight (enabled = false)"]

Runtime startup

A small program against a4e01eb ran the adapter startup sequence (Settings::from_toml, validate_settings_for_runtime, compile_auction_plan, IntegrationRegistry::with_plan) on the same cases. All start, and the registry shows the effect:

A1 typo [integrations.prebdi] (meant prebid)
    runtime: ACCEPTED; registered=[]; `prebid` registered: false
D1 sourcepoint `enable = true` (typo of enabled)
    runtime: ACCEPTED; registered=[]; `sourcepoint` registered: false
D3 gpt `rewrite_scripts = false` (testlight's spelling; gpt reads rewrite_script)
    runtime: ACCEPTED; registered=["gpt"]; `gpt` registered: true
    effective gpt.rewrite_script = true
D4 google_tag_manager `cache_ttl_seconds = 60` (other integrations' name; GTM reads cache_max_age)
    runtime: ACCEPTED; registered=["google_tag_manager"]; `google_tag_manager` registered: true
    effective google_tag_manager.cache_max_age = 900
D5 gpt `enable = false` (typo of enabled; gpt defaults to enabled)
    runtime: ACCEPTED; registered=["gpt"]; `gpt` registered: true
D6 didomi `enable = false` (typo of enabled; didomi defaults to enabled)
    runtime: ACCEPTED; registered=["didomi"]; `didomi` registered: true

ts config push stores serde_json::to_value of the typed config (build_config_envelope in EdgeZero v0.0.8, crates/edgezero-cli/src/config.rs:1453-1466). That value keeps the unknown keys:

integrations in serialized blob data: {"gpt":{"enabled":true,"rewrite_scripts":false},"kargo":{"enabled":true,"publisher_id":"your-kargo-publisher-id"}}

Expected behavior

• An unknown integration ID fails ts config validate and ts config push with an error that names it and lists the known IDs.
• An unknown key in any integration table fails validation with an error that names the key, the same way a top-level typo does today.
• Runtime startup never ignores these silently. At minimum it logs a warning.
• The configuration guide's statement about strict key validation is true.

Actual behavior

• Unknown IDs and unknown keys in ten integration tables validate, push and start with no diagnostic.
• An integration whose enabled key is misspelled stays at its default. That is off for adserver_mock, google_tag_manager, js_asset_proxy, nextjs, sourcepoint and testlight, and on for didomi, gpt, lockr and permutive. So enable = false does not turn off GPT or Didomi.
• Where an integration does reject an unknown key, the CLI message says only "configuration could not be parsed" and does not name the key.

Root cause

[integrations] is a raw JSON map (crates/trusted-server-core/src/settings.rs:218-222):

pub struct IntegrationSettings {
    #[serde(flatten)]
    entries: HashMap<String, JsonValue>,
}

Typed parsing happens only in IntegrationSettings::get_typed (settings.rs:314-361), which callers reach with a fixed ID. Validation calls it for a fixed list of 14 IDs (config.rs:301-330) plus js_asset_proxy (config.rs:280-299). No code looks at the keys that are left over. The only iteration over entries is the Debug impl (settings.rs:224-233).

Inside a known table, serde ignores unknown fields unless the struct opts out. Integration config structs at a4e01eb (paths under crates/trusted-server-core/src/integrations/):

LegacyApsProviderConfig (aps.rs:154) and LegacyPrebidServerConfig (prebid.rs:338) also implement IntegrationConfig but are #[cfg(test)] only.

Nested tables are covered where they exist, and only the top level of js_asset_proxy is open:

• Prebid: PrebidManagedUserIdConfig (prebid.rs:323), PrebidManagedUserIdStorage (prebid.rs:262), PrebidBundleBuildConfig (prebid.rs:497) and PrebidBundleModulesConfig (prebid.rs:509) all deny. managed_user_ids[].params is an open map by design.
• DataDome: ProtectionTestBypassConfig (datadome.rs:133) and ProtectionIpCidrSourceConfig (datadome/protection_scope.rs:19) deny. ProtectionExclusionRuleConfig (protection_scope.rs:32) deliberately does not, because it flattens the internally tagged ProtectionMatcherConfig (protection_scope.rs:50), which does. A probe confirmed that a rule containing enable = false is still rejected. client_side_configuration is open JSON by design.
• js_asset_proxy: each JsAssetProxyAsset (js_asset_proxy.rs:55) denies, but the top-level JsAssetProxyConfig does not.
• adserver_mock.context_query_params is an open map by design.
• The other nine lenient structs have no nested structs.

Disabled blocks follow the same rule. get_typed validates an explicitly disabled block by deserializing the full schema and tolerating only missing field errors (settings.rs:247-253, 326-333), so an unknown key is rejected only for the five structs that deny. A block whose enabled key is misspelled is not "explicitly disabled" at all, so it is parsed with its default enabled value (settings.rs:335-350).

The CLI message does not name the key because report_to_validation_error (config.rs:504-511) uses report.to_string(), which prints only the outer context. get_typed wraps the serde error in "Integration '{id}' configuration could not be parsed" (settings.rs:327-331, 335-341). validate_js_asset_proxy_config already includes the serde text (config.rs:284-291), which is why its errors name the key.

Impact

• Every deployment and adapter. Nothing is logged, so an operator finds out only by noticing that a feature is missing.
• Likely causes are typos, near-miss names between integrations (rewrite_script in GPT, rewrite_scripts in Testlight, rewrite_sdk elsewhere; cache_max_age in GTM, cache_ttl_seconds elsewhere), and blocks copied from the Lockr, Kargo and GAM guides.
• The effect is a misconfiguration, not a crash. Its severity depends on the setting that was lost. The worst case is an operator trying to switch off a default-on integration (Didomi, GPT, Lockr, Permutive) during an incident with a misspelled enabled: validation passes and the integration keeps running.
• Existing deployments may already hold stale blocks. The raw map has been accepted since Proposal to standardize integrations #113, and ts config push stores it verbatim, so a stored blob can carry [integrations.kargo] or a misspelled key today. Any fix that fails runtime startup on these would take such a service down when the new binary deploys.

Proposed fix

1. Add #[serde(deny_unknown_fields)] to the ten structs in the table.
2. In validate_settings_for_deploy (config.rs:247), reject any [integrations] key that is not a known ID. Name it and list the known IDs. DEPLOY_VALIDATED_INTEGRATION_IDS (config.rs:39-55, test-only today) already holds the 15 IDs and can become the non-test source.
3. In validate_settings_for_runtime (config.rs:268), log a warning for an unknown ID and keep starting. Do not fail startup for an unknown ID.
4. Include the serde error text in the get_typed context, as validate_js_asset_proxy_config does, so the CLI names the offending key.
5. Fix the docs: the Lockr guide's config block, the "Strict Key Validation" paragraph, and the [integrations.custom] example in docs/guide/error-reference.md:391, which would now be rejected.
6. Add the attribute to TesterCookieConfig and TinybirdSettings in the same change, or track them separately, so that the guide's statement holds for every section.

Steps 1 to 3 were applied to a copy of a4e01eb. Both new tests pass, all 2763 trusted-server-core lib tests pass on the host target (including the test that uncomments the documented template blocks), and the trusted-server-cli unit and integration tests pass. None of the 24 tracked .toml files uses a key the change rejects. In the docs, only the Lockr guide block and the [integrations.custom] example would be rejected, which step 5 covers.

Compatibility: step 1 also applies at runtime, because the runtime parses the same structs. A stored blob with an unknown key in one of the ten tables would fail startup on the new binary. The release note should give this order:

1. Run the new ts config validate against the current config.
2. Remove what it reports. Removing keys is safe for the old binary.
3. Push, then deploy the new binary.

The retired Prebid and APS fields went through the same kind of documented break in #1016. If maintainers want no startup risk at all, the alternative is a warn-first release: collect ignored keys at runtime and log them, while push rejects them. Then reject at runtime in a later release. That needs a way to collect ignored keys, such as the serde_ignored crate, which is not a dependency today.

Done when

• The ten structs above deny unknown fields, and deploy_validation_rejects_unknown_integration_fields passes for enabled and disabled blocks.
• deploy_validation_rejects_unknown_integration_id passes, and the error lists the known IDs.
• A runtime test shows an unknown integration ID produces a warning and startup still succeeds.
• ts config validate names the offending key for an unknown field in any integration table.
• [tester_cookie] and [tinybird] reject unknown keys, or a follow-up issue tracks them.
• docs/guide/integrations/lockr.md, the "Strict Key Validation" paragraph in docs/guide/configuration.md and docs/guide/error-reference.md match the new behavior, and the release notes carry the upgrade order.
• CI gates pass, including cargo test-fastly.

Affected area

Integrations (prebid, lockr, permutive, etc.)

Version

main at a4e01eb

Related

• Open the integration seam so a vendor module can live outside core #1094 (open, part of the Add the integration seam design specs #1084 stack): adds deny_unknown_fields to the same ten structs. It does not reject unknown integration IDs.
• Define integrations crate split and ordered configuration #1194 (draft spec): plans to reject unknown IDs in a new config schema. This issue is about today's schema.
• Add full-surface documentation refresh spec #1049 (merged at 4c6d26a): turned kargo.md and gam.md into status pages. Before it, both showed blocks that validated and did nothing.
• Add configuration-driven OpenRTB auction providers #1016 (merged): precedent for a documented break. It made retired Prebid and APS fields fail through deny_unknown_fields.
• Document the config template and harden config validation #870 (merged): hardened deploy validation for template placeholders. It did not cover unknown keys.
• Configuration errors silently disable integrations #408 (closed): parse errors were swallowed with .ok(), silently disabling integrations. Same symptom, different mechanism.