IO_CONTRACTS.md — living contract for I/O and control-plane semantics #1303

Activity

1. strawgate

   added

   documentationImprovements or additions to documentationImprovements or additions to documentation

   P2: mediumMinor bug, missing validation, metric inaccuracyMinor bug, missing validation, metric inaccuracy

   architectureSystemic design improvements that eliminate classes of bugsSystemic design improvements that eliminate classes of bugs

   on Apr 6, 2026
2. github-actions commented on Apr 6, 2026

   

   github-actionsBot

   on Apr 6, 2026 – with GitHub Actions

   Contributor

   More actions

   tl;dr: This is needs-decision because the issue’s checkpoint rule (never on reject) conflicts with current implemented + verified behavior (reject advances checkpoint), so we need one explicit policy decision before drafting dev-docs/IO_CONTRACTS.md as a normative contract.

   Recommendation

   Pick Option 1 below (keep current reject semantics, document it explicitly), then implement IO_CONTRACTS.md mechanically.

   Findings

   Code evidence (paths + lines)

   1. Receiver→Pipeline backpressure is already explicit and enforced

      • crates/logfwd-io/src/otlp_receiver.rs:25 (CHANNEL_BOUND: 4096)
      • crates/logfwd-io/src/otlp_receiver.rs:59 (mpsc::sync_channel(capacity))
      • crates/logfwd-io/src/otlp_receiver.rs:230-236 full channel => HTTP 429
      • crates/logfwd-io/src/otlp_receiver.rs:1043 test returns_429_when_channel_full_not_200

   2. Receiver field names are currently hardcoded (not semantic-fields-driven yet)

      • crates/logfwd-io/src/otlp_receiver.rs:363,371,379,403,409,496,504,511,532,539
        (timestamp_int, level, message, trace_id, span_id)

   3. Pipeline→Sink contract today is RecordBatch + BatchMetadata; checkpoint state is separate ticket state

      • crates/logfwd-output/src/sink.rs:74-80 (Sink::send_batch(&RecordBatch, &BatchMetadata))
      • crates/logfwd-output/src/lib.rs:75-80 (BatchMetadata { resource_attrs, observed_time_ns })
      • crates/logfwd/src/pipeline.rs:917-930 metadata creation + submit to worker pool

   4. Checkpoint behavior conflict (core decision point)

      • crates/logfwd/src/pipeline.rs:977-982 failed output path uses ticket.reject() then machine.apply_ack(receipt)
      • crates/logfwd-types/src/pipeline/lifecycle.rs:524-532 unit test reject_advances_checkpoint
      • crates/logfwd-types/src/pipeline/lifecycle.rs:1069-1089 proof comment + verification for reject-advances
      • tla/PipelineMachine.tla:153,173 RejectBatch(s,b) == AckBatch(s,b)

   5. Delivery semantics are documented, but not in one boundary contract doc yet

      • book/src/architecture/pipeline.md:116 at-least-once with queue
      • dev-docs/SCANNER_CONTRACT.md:1-5 scanner-only scope; does not cover full I/O boundaries

   6. Related roadmap context is active

      • research: I/O component refactor — library evaluation and benchmarking #1283 (I/O refactor research)
      • Shared semantic field definitions (logfwd-types) #1302 (shared semantic field definitions)
      • Open PR [codex] add OTLP and file IO contract oracles #1290 ([codex] add OTLP and file IO contract oracles) likely overlaps this area

   Verification

   Commands run

   $ cargo test -q -p logfwd-io returns_429_when_channel_full_not_200 -- --nocapture
   running 1 test
   .
   test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 41 filtered out; finished in 0.00s
   
   $ cargo test -q -p logfwd-types reject_advances_checkpoint -- --nocapture
   running 1 test
   .
   test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 182 filtered out; finished in 0.01s

   Needs human decision

   What’s unclear: Should checkpoint advancement occur on permanent sink rejection?

   1. Option 1 (recommended): keep current behavior

      • Rule: reject advances checkpoint (drop undeliverable data, avoid infinite replay/drain stalls)
      • Pros: matches current code, tests, and TLA model; no behavior change required
      • Cons: explicitly accepts at-most-once for rejected batches

   2. Option 2: change behavior to “never advance on reject”

      • Rule: keep checkpoint behind rejected batch
      • Pros: stricter loss-avoidance interpretation
      • Cons: can stall drain/replay indefinitely on permanently bad records; requires coordinated code/spec/test/model rewrite

   3. Option 3: configurable policy (advance_on_reject = true|false)

      • Pros: explicit operator control
      • Cons: bigger design surface + combinatorial test/spec burden

   Recommendation: Option 1.

   Label: please add needs-decision.

   Detailed action plan (after decision)

   Mechanical implementation steps

   1. Create dev-docs/IO_CONTRACTS.md

      • Add sections:
        • Receiver → Pipeline
        • Pipeline → Sink
        • Checkpoint rules
        • Error classification
        • File lifecycle
      • Use a single matrix schema per row:
        • Boundary | Guarantee | Preconditions | Failure mode | Delivery semantics | Duplicate/loss window | Checkpoint rule | Enforcement mechanism | Evidence (path:line) | TODO

   2. Cross-link scanner scope in dev-docs/SCANNER_CONTRACT.md (top section)

      • Add explicit note: scanner-only contract
      • Link to dev-docs/IO_CONTRACTS.md for boundary-level guarantees

   3. Populate initial enforcement column

      • Kani / proptest / TLA+ / integration test where evidence exists
      • TODO: not yet enforced where it does not

   4. Resolve semantic field naming row with Shared semantic field definitions (logfwd-types) #1302 state

      • Document current hardcoded names as “current behavior”
      • Mark semantic-fields centralization as pending until Shared semantic field definitions (logfwd-types) #1302 lands

   5. Conflict check with open PR [codex] add OTLP and file IO contract oracles #1290 before merge

      • Reconcile overlapping contract/oracle wording and references

   6. Tests/docs checks for triage handoff

      • cargo test -p logfwd-io --test it
      • cargo test -p logfwd-types reject_advances_checkpoint

   Related items

   Type	Link	Relevance
   Issue	#1303	This triage target
   Issue	#1283	I/O refactor research context
   Issue	#1302	Semantic field constants dependency
   PR	#1290	Open work on OTLP/file I/O contract oracles; likely overlap
   File	crates/logfwd-io/src/otlp_receiver.rs	Receiver boundary guarantees (429/backpressure + field names)
   File	crates/logfwd/src/pipeline.rs	Ticket ack/reject + checkpoint advancement path
   File	crates/logfwd-types/src/pipeline/lifecycle.rs	Ordered ACK + reject semantics tests/proofs
   File	tla/PipelineMachine.tla	Formal reject==ack transition model
   File	dev-docs/SCANNER_CONTRACT.md	Existing scanner-only contract scope
   File	book/src/architecture/pipeline.md	Delivery-semantics narrative

   Note

   🔒 Integrity filtering filtered 6 items

   Integrity filtering activated and filtered the following items during workflow execution.
   This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.

   • issue:IO_CONTRACTS.md — living contract for I/O and control-plane semantics #1303 (issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
   • #638 (search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
   • pr:[codex] add OTLP and file IO contract oracles #1290 (pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
   • issue:research: I/O component refactor — library evaluation and benchmarking #1283 (issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
   • issue:Shared semantic field definitions (logfwd-types) #1302 (issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)
   • pr:IO_CONTRACTS.md — living contract for I/O and control-plane semantics #1303 (pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)

   ---

   What is this? | From workflow: Issue Triage

   Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
3. strawgate

   changed the title [-]IO_CONTRACTS.md — living contract document for I/O boundaries[/-] [+]IO_CONTRACTS.md — living contract for I/O and control-plane semantics[/+] on Apr 6, 2026
4. coderabbitai

   added

   enhancementNew feature or requestNew feature or request

   on Apr 6, 2026
5. strawgate commented on Apr 6, 2026

   

   strawgate

   on Apr 6, 2026

   OwnerAuthor

   More actions

   Scheduling layer for this issue:

   1. work-unit: control-plane contracts — probe, status, and roll-up semantics #1412 — contract completion for probe, status, and roll-up semantics
   2. work-unit: control plane — process phase, drain, and probe truth #1413 — process phase, drain, and probe truth
   3. work-unit: control plane — shared status model and admin status surface #1414 — shared status snapshots and /admin/v1/status
   4. work-unit: control plane — input lifecycle status wiring #1415 — input lifecycle status wiring for receivers and tailers
   5. work-unit: control plane — output lifecycle status wiring #1416 — sink lifecycle, degradation, and failure wiring

   This keeps #1303 as the contract source of truth while letting execution happen in bounded, merge-safe slices.
6. strawgate

   added

   component:diagnosticsDiagnostics endpoints, dashboard, health, probes, and trace explorerDiagnostics endpoints, dashboard, health, probes, and trace explorer

   on Apr 6, 2026
7. strawgate commented on Apr 11, 2026

   

   strawgate

   on Apr 11, 2026

   OwnerAuthor

   More actions

   Closing as completed via the canonical contract document and follow-on work-units.

   Verification evidence:

   • Contract doc exists and is treated as canonical for I/O + control-plane boundaries: (scope includes , , , health vocabulary, readiness rules, roll-ups, verification mapping).
   • Planned execution slices were landed and closed: work-unit: control-plane contracts — probe, status, and roll-up semantics #1412, work-unit: control plane — process phase, drain, and probe truth #1413, work-unit: control plane — shared status model and admin status surface #1414, work-unit: control plane — input lifecycle status wiring #1415, work-unit: control plane — output lifecycle status wiring #1416.

   Note: issue text names , but implementation settled on as the canonical file.
8. strawgate

   closed this as completedon Apr 11, 2026
9. strawgate commented on Apr 11, 2026

   

   strawgate

   on Apr 11, 2026

   OwnerAuthor

   More actions

   Follow-up verification summary (clean copy):

   • Canonical contract doc exists and is actively used: dev-docs/ADAPTER_CONTRACT.md.
   • It covers the contract scope requested here, including /live, /ready, /admin/v1/status semantics, health vocabulary, readiness behavior, roll-up rules, and verification mapping.
   • The scoped control-plane work-units were delivered and closed: work-unit: control-plane contracts — probe, status, and roll-up semantics #1412, work-unit: control plane — process phase, drain, and probe truth #1413, work-unit: control plane — shared status model and admin status surface #1414, work-unit: control plane — input lifecycle status wiring #1415, work-unit: control plane — output lifecycle status wiring #1416.

   The original issue filename text used IO_CONTRACTS.md; implementation standardized on ADAPTER_CONTRACT.md.