bug: generator inputs accept input.path and ignore it silently #1077

Activity

1. strawgate

   added

   bugSomething isn't workingSomething isn't working

   on Apr 5, 2026
2. github-actions commented on Apr 5, 2026

   

   github-actionsBot

   on Apr 5, 2026 – with GitHub Actions

   Contributor

   More actions

   tl;dr: input.type: generator currently accepts input.path and silently ignores it; the safest fix is to reject path during config validation in logfwd-config.

   Recommendation

   Reject path for generator in Config::validate and add a regression test in crates/logfwd-config/src/lib.rs so this cannot regress.

   Findings

   Key files

   • crates/logfwd-config/src/lib.rs:233-245 — InputConfig has shared fields for all input kinds, including path: Option(String).
   • crates/logfwd-config/src/lib.rs:643-659 — validation requires path for file, listen for udp/tcp, but does nothing for generator (InputType::Otlp | InputType::Generator | InputType::ArrowIpc => {}).
   • crates/logfwd/src/pipeline.rs:286-299 — pipeline pre-resolves/canonicalizes input_cfg.path when present.
   • crates/logfwd/src/pipeline.rs:1299-1318 — generator runtime construction reads listen and format, not path.
   • book/src/config/inputs.md:23-31 — generator docs only expose listen.
   • book/src/config/reference.md:141 — generator says “No extra fields required.”

   This combination means path is accepted by schema and passes validation for generator, but has no runtime effect.

   Verification

   I validated current behavior with:

   export RUSTC_WRAPPER=
   cargo run -q -p logfwd -- --config /tmp/gh-aw/agent/generator-path.yaml --validate
   cargo test -q -p logfwd-config all_input_types -- --nocapture

   Where /tmp/gh-aw/agent/generator-path.yaml contains:

   input:
     type: generator
     path: /tmp/should-not-be-used.log
   output:
     type: stdout

   Output:

   ready: default
   config ok: 1 pipeline(s)
   VALIDATE_EXIT=0
   
   running 1 test
   .
   test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 56 filtered out; finished in 0.00s
   

   So generator+path validates successfully today.

   Detailed Action Plan

   1. In crates/logfwd-config/src/lib.rs inside Config::validate input loop (~643+), add a generator-specific guard:
      • if input.input_type == InputType::Generator && input.path.is_some() -> return ConfigError::Validation("...generator...does not support 'path'...").
   2. Add regression test near existing validation tests in crates/logfwd-config/src/lib.rs (around validation_missing_input_path / all_input_types):
      • YAML with input.type: generator + path: ... + output.type: stdout.
      • Assert Config::load_str returns error containing both generator and path.
   3. Update user docs only if needed for clarity:
      • book/src/config/inputs.md generator table can explicitly say path is invalid for generator.
      • book/src/config/reference.md keep “No extra fields required” and optionally add “path is file-only”.
   4. Run targeted checks:
      • cargo test -p logfwd-config (or at least the new test + nearby validation tests).

   Related Items

   Type	Link / Path	Relevance
   Issue ref in code	crates/logfwd-config/src/lib.rs:668,1691 (#696)	Prior fix for “silently ignored” config behavior pattern
   Issue ref in code	crates/logfwd-config/src/lib.rs:1866,2010 (#540)	Prior config strictness regression coverage
   PRs currently open in repo	#1027, #1003, #1000	Listed from MCP; none obviously in this config/input area by title
   Open PRs in this area	N/A	I could list open PRs, but MCP integrity policy blocked reading per-PR changed-file lists, so I could not verify direct file overlap
   Affected crates	logfwd-config (primary), logfwd (runtime evidence), docs in book/	Validation fix is in logfwd-config; runtime behavior stays unchanged once invalid config is rejected
   Test patterns to follow	crates/logfwd-config/src/lib.rs:1064+ (validation_missing_input_path), :1271+ (all_input_types)	Best nearby style for regression test

   Note

   🔒 Integrity filtering filtered 8 items

   Integrity filtering activated and filtered the following items during workflow execution.
   This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.

   • resource:search_pull_requests (search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
   • issue:strawgate/memagent#unknown (search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
   • issue:bug: generator inputs accept input.path and ignore it silently #1077 (issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
   • issue:file input: max_open_files: 0 silently disables all file reading #696 (issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
   • issue:Enrichment types k8s_path, host_info, static, csv, jsonl have code but no config wiring #540 (issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
   • pr:perf: add throughput ceiling benchmark (generator → scan → null) #1027 (pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
   • pr:Convert JsonLinesSink to async reqwest (migrate from ureq) #1000 (pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
   • pr:adopt thiserror 2.0 across workspace, define per-crate error enums #1003 (pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)

   ---

   What is this? | From workflow: Issue Triage

   Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
3. strawgate commented on Apr 5, 2026

   

   strawgate

   on Apr 5, 2026

   OwnerAuthor

   More actions

   Validation: CONFIRMED — shared root cause with all silent-ignore config issues

   Both InputConfig and OutputConfig are single flat structs (#[serde(deny_unknown_fields)]) with fields shared across all input/output types. Config::validate() checks that required fields are present for each type but does not reject extraneous optional fields that have no meaning for the selected type.

   This bug is one of ~12 confirmed instances of the same pattern (see also #1057, #1058, #1060, #1064, #1065, #1066, #1070, #1071, #1072, #1073, #1075, #1077, #1078).

   Root cause confirmed:

   • crates/logfwd-output/src/lib.rs: runtime builders use match cfg.output_type and silently ignore fields not consumed by the selected arm.
   • crates/logfwd-config/src/lib.rs Config::validate(): validates required fields per type but has no reject-if-present checks for inapplicable optional fields.

   Unified fix — add per-type field rejection to Config::validate():

   // Example for OutputConfig:
   match output.output_type {
       OutputType::Stdout | OutputType::Null => {
           if output.endpoint.is_some() { warn_or_err("endpoint not used by this output type"); }
           if output.index.is_some() { warn_or_err("index not used by this output type"); }
           if output.compression.is_some() { warn_or_err("compression not used by this output type"); }
           if output.auth.is_some() { warn_or_err("auth not used by this output type"); }
           // etc.
       }
       OutputType::Elasticsearch => {
           if output.protocol.is_some() { warn_or_err("protocol not used by elasticsearch"); }
           // etc.
       }
       // ...
   }

   A single pass over all output types covering all optional fields fixes all 12 related issues at once.
4. strawgate

   added 3 commits that reference this issue on Apr 5, 2026

   fix(config): reject type-inappropriate fields in input/output validation

   ...

   5dfde4a

   fix(config): reject type-inappropriate fields in input/output validation

   ...

   802bdb3

   fix(config): reject type-inappropriate fields in input/output validation

   ...

   2d249da
5. strawgate commented on Apr 5, 2026

   

   strawgate

   on Apr 5, 2026

   OwnerAuthor

   More actions

   Fixed on master in PR #1123 (923a033bd8e855f4011f8ad9d5248b9e44a3546a).

   I rechecked origin/master; these type-inappropriate config fields are now rejected during validation, so this issue/work-unit looks stale and safe to close.
6. strawgate

   closed this as completedon Apr 5, 2026