bug: tcp/udp inputs accept max_open_files and ignore it silently #1075

Activity

1. strawgate

   added

   bugSomething isn't workingSomething isn't working

   on Apr 5, 2026
2. github-actions commented on Apr 5, 2026

   

   github-actionsBot

   on Apr 5, 2026 – with GitHub Actions

   Contributor

   More actions

   max_open_files is currently accepted for tcp/udp and ignored at runtime; the fix should be a config-validation guard in logfwd-config that restricts it to file inputs.

   Recommendation

   Reject max_open_files whenever input_type != file (at least tcp/udp; ideally also generator/otlp/arrow_ipc for consistency with #1072), while keeping the existing max_open_files >= 1 check for file.

   Key files

   • crates/logfwd-config/src/lib.rs:233-245 — InputConfig defines max_open_files: Option(usize) with docstring “Applies only to file inputs.”
   • crates/logfwd-config/src/lib.rs:594-674 — Config::validate() input validation; currently only rejects max_open_files == 0 (:668-673), regardless of input type.
   • crates/logfwd/src/pipeline.rs:1269-1362 — build_input_state(...); max_open_files is consumed only in InputType::File (:1282-1287), not in udp/tcp branches (:1331-1361).
   • crates/logfwd-io/src/tcp_input.rs:79-81 — TcpInput::new(name, addr) signature has no open-file parameter.
   • crates/logfwd-io/src/udp_input.rs:34-35 — UdpInput::new(name, addr) signature has no open-file parameter.
   • crates/logfwd-io/src/tail.rs:132-160,963-964 — TailConfig.max_open_files and LRU eviction usage (file-only behavior).

   Findings

   1. The config layer exposes max_open_files on all input objects because InputConfig is shared across input types (crates/logfwd-config/src/lib.rs:233-245).
   2. Validation currently enforces only lower-bound (!= 0) and does not enforce applicability by input type (crates/logfwd-config/src/lib.rs:668-673).
   3. Runtime behavior confirms silent ignore for network inputs: pipeline wiring only passes max_open_files into TailConfig in the file branch (crates/logfwd/src/pipeline.rs:1282-1287).
   4. Existing validation pattern for “field only valid for specific type” already exists (request_mode is only supported for elasticsearch outputs, crates/logfwd-config/src/lib.rs:734-737), which is a good template for this fix.
   5. Docs currently list only path under file input fields and do not mention max_open_files (book/src/config/reference.md:83-90), so docs should be updated to avoid ambiguity.

   Verification

   I ran targeted CLI/config checks locally:

   $ cargo run -q -p logfwd -- --config (tcp-with-max_open_files).yaml --validate
   TCP_VALIDATE_EXIT=0
     ready: default
   config ok: 1 pipeline(s)
   
   $ cargo run -q -p logfwd -- --config (udp-with-max_open_files).yaml --validate
   UDP_VALIDATE_EXIT=0
     ready: default
   config ok: 1 pipeline(s)
   
   $ cargo run -q -p logfwd -- --validate --config (file-max_open_files-0).yaml
   FILE_ZERO_EXIT=1
   error: config validation error: pipeline 'default' input '#0': max_open_files must be at least 1
   
   $ RUSTC_WRAPPER= cargo test -q -p logfwd-config max_open_files -- --nocapture
   running 3 tests
   ...
   test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 54 filtered out

   Detailed Action Plan

   1. In crates/logfwd-config/src/lib.rs inside Config::validate() input loop (~638+), split max_open_files validation by input.input_type:
      • file: keep current == 0 rejection.
      • non-file: if max_open_files.is_some(), return validation error like:
        • pipeline '{name}' input '{label}': max_open_files is only supported for file inputs.
   2. Add config tests in crates/logfwd-config/src/lib.rs near existing max_open_files_* tests (:1695+):
      • tcp + max_open_files => error.
      • udp + max_open_files => error.
      • (recommended bundle with bug: generator/otlp inputs accept max_open_files and ignore it silently #1072) generator and otlp + max_open_files => error.
   3. Keep existing positive file tests (max_open_files_one_accepted, absent accepted) unchanged.
   4. Add/adjust one integration-level validation case in crates/logfwd/tests/it/integration.rs near test_config_validation_errors (:382+) if maintainers want CLI-surface coverage beyond unit tests.
   5. Update book/src/config/reference.md under file input fields (:83-90) to explicitly document max_open_files as file-only with default 1024.

   Open PRs in this area

   Using local branch diff against fetched open PR heads:

   PR	Status	Overlap with relevant files
   #1003 (copilot/adopt-thiserror-2-0)	Open	Yes — touches crates/logfwd-config/src/lib.rs and crates/logfwd/src/pipeline.rs
   #1000 (copilot/convert-jsonlines-sink-to-reqwest)	Open	No overlap found with logfwd-config/input files above
   #1027 (copilot/perf-find-throughput-ceiling)	Open	No overlap found with logfwd-config/input files above

   Related Items

   Type	Link	Relevance
   Issue	#1072 (open)	Same bug class for non-file inputs (generator/otlp) accepting max_open_files
   Issue	#696 (closed)	Added current lower-bound validation (max_open_files >= 1) for file inputs
   Issue	#193 (closed)	Original feature introducing file tailer max_open_files behavior
   File	crates/logfwd-config/src/lib.rs	Validation rule should be tightened here
   File	crates/logfwd/src/pipeline.rs	Confirms max_open_files is file-branch-only at runtime
   File	book/src/config/reference.md	Needs explicit max_open_files file-only documentation

   Note

   🔒 Integrity filtering filtered 6 items

   Integrity filtering activated and filtered the following items during workflow execution.
   This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.

   • issue:bug: tcp/udp inputs accept max_open_files and ignore it silently #1075 (issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
   • issue:file input: max_open_files: 0 silently disables all file reading #696 (issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
   • resource:search_pull_requests (search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
   • pr:Convert JsonLinesSink to async reqwest (migrate from ureq) #1000 (pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
   • pr:perf: add throughput ceiling benchmark (generator → scan → null) #1027 (pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
   • pr:adopt thiserror 2.0 across workspace, define per-crate error enums #1003 (pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)

   ---

   What is this? | From workflow: Issue Triage

   Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
3. strawgate commented on Apr 5, 2026

   

   strawgate

   on Apr 5, 2026

   OwnerAuthor

   More actions

   Validation: CONFIRMED — shared root cause with all silent-ignore config issues

   Both InputConfig and OutputConfig are single flat structs (#[serde(deny_unknown_fields)]) with fields shared across all input/output types. Config::validate() checks that required fields are present for each type but does not reject extraneous optional fields that have no meaning for the selected type.

   This bug is one of ~12 confirmed instances of the same pattern (see also #1057, #1058, #1060, #1064, #1065, #1066, #1070, #1071, #1072, #1073, #1075, #1077, #1078).

   Root cause confirmed:

   • crates/logfwd-output/src/lib.rs: runtime builders use match cfg.output_type and silently ignore fields not consumed by the selected arm.
   • crates/logfwd-config/src/lib.rs Config::validate(): validates required fields per type but has no reject-if-present checks for inapplicable optional fields.

   Unified fix — add per-type field rejection to Config::validate():

   // Example for OutputConfig:
   match output.output_type {
       OutputType::Stdout | OutputType::Null => {
           if output.endpoint.is_some() { warn_or_err("endpoint not used by this output type"); }
           if output.index.is_some() { warn_or_err("index not used by this output type"); }
           if output.compression.is_some() { warn_or_err("compression not used by this output type"); }
           if output.auth.is_some() { warn_or_err("auth not used by this output type"); }
           // etc.
       }
       OutputType::Elasticsearch => {
           if output.protocol.is_some() { warn_or_err("protocol not used by elasticsearch"); }
           // etc.
       }
       // ...
   }

   A single pass over all output types covering all optional fields fixes all 12 related issues at once.
4. strawgate

   added 3 commits that reference this issue on Apr 5, 2026

   fix(config): reject type-inappropriate fields in input/output validation

   ...

   5dfde4a

   fix(config): reject type-inappropriate fields in input/output validation

   ...

   802bdb3

   fix(config): reject type-inappropriate fields in input/output validation

   ...

   2d249da
5. strawgate commented on Apr 5, 2026

   

   strawgate

   on Apr 5, 2026

   OwnerAuthor

   More actions

   Fixed on master in PR #1123 (923a033bd8e855f4011f8ad9d5248b9e44a3546a).

   I rechecked origin/master; these type-inappropriate config fields are now rejected during validation, so this issue/work-unit looks stale and safe to close.
6. strawgate

   closed this as completedon Apr 5, 2026