fix(transforms): log the compression failures that silently become "no savings"

[chopratejas]

Description

Five places in the compression path caught every exception and returned the input unchanged, with no log line. A compression bug therefore looked exactly like content that did not compress. Each now logs a WARNING that names the exception type and counts, and a DEBUG line that names the exception types and code locations (describe_exception). No level logs the exception text or a traceback, because compressor and store errors can quote tool output or credentials. Return values do not change.

Refs #4136

Depends on #4138 (headroom/log_safety.py); its files show in this diff until it merges.

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update
• Performance improvement
• Code refactoring (no functional changes)

Changes Made

• compression_batches.compress_batch_with_router: warn when router.compress raises (the batch still passes through).
• cross_turn_dedup.dedup_blocks: warn on failure. The "error": True stat it returned was never read.
• lossless_compaction.compact_lossless: warn when a fold raises, with the kind and length. It still never raises.
• kompress_compressor.store_kompress_in_ccr: warn on a CCR store failure. The output then has no retrieval marker.
• smart_crusher CCR mirror: a store write failure is now WARNING, not DEBUG.
• Each WARNING carries type(exc).__name__ plus counts or the hash, and never the exception text. The detail is a separate DEBUG line built with describe_exception(e), which gives types and code locations and never the message or traceback (unless the operator sets HEADROOM_DEBUG_DUMP=full). It only runs when DEBUG is enabled.
• New tests/test_transforms_failure_logging.py: one parametrized test over all five paths. It checks pass-through, one WARNING and one DEBUG detail, and formats every captured record with logging.Formatter to assert a content canary and a credential canary (raised in the exception and its cause) are absent. All five fail on main, and all five fail on the earlier exc_info=True version of this PR. A second parametrized test checks describe_exception is not called when DEBUG is off. A third runs the real compact_lossless fallback with DEBUG on and an OSError whose errno is out of the C int range (2**31, and 10**10000 built at runtime), with the integer digit limit on and off. It checks the original content comes back and the DEBUG line stays bounded.

Testing

• Unit tests pass (pytest)
• Linting passes (ruff check .)
• Type checking passes (mypy headroom)
• New tests added for new functionality
• Manual testing performed

Test Output

$ pytest tests/test_transforms_failure_logging.py   # with main's product code
5 failed
$ pytest tests/test_transforms_failure_logging.py tests/test_lossless_mode.py tests/test_cross_turn_dedup.py tests/test_compression_batches.py tests/test_kompress_marker_gate.py tests/test_garbled_compression_fixes.py tests/test_ccr.py tests/test_lossless_excluded_compaction.py tests/test_lossless_first_dispatch.py tests/test_bash_search_lossless_fold.py tests/test_lossless_timestamp_guard.py tests/test_remote_kompress_dropin.py tests/test_ccr_marker_policy.py tests/test_responses_cross_turn_dedup.py tests/test_proxy_handlers_batch.py
258 passed in 24.04s
$ ruff check / ruff format (0.16.8): All checks passed;  mypy (5 modules): Success

Real Behavior Proof

• Environment: macOS, Python 3.12, repo .venv, this branch's worktree with the prebuilt headroom._core. Temp HOME, HEADROOM_WORKSPACE_DIR and CODEX_HOME; nothing touched the real home.
• Exact command / steps: (1) shared smoke harness: a real proxy from this checkout against a local fake upstream, with one Anthropic and one OpenAI request carrying a compressible tool result (smoke.sh <worktree>); (2) forced failure: compact_lossless with collapse_runs patched to raise an error whose text contains a fake token, logging at INFO.
• Observed result: The proxy serves both requests (200) with savings. The forced failure returns the content unchanged and logs one WARNING naming RuntimeError, without the token text. Output below.
• Not tested: A live proxy request that hits one of the five failure paths: they need an injected bug, which the direct check and the unit tests supply.

branch: cleanup/transforms-log-silent-failures @ f2557723
health: 200
POST /v1/messages        -> HTTP200 smoke-ok
POST /v1/chat/completions -> HTTP200 smoke-ok
stats: requests=2 failed=0 tokens_saved=13574 savings_percent=99.82
proxy.log WARNING/ERROR lines ($SMOKE/ws/logs/proxy-64662.log):
   2 headroom.proxy - WARNING - event=proxy_offline_mode air-gap active — every connection Headroom initiates is refused (telemetry, update check, license reporter, model/tokenizer/binary/dataset downloads, remote Kompress, OTLP/Langfuse export, Copilot auth, subscription polling, OpenAI embedders, Headroom Cloud compression). Still allowed on purpose: forwarding your requests to the upstream you configured, operator-configured local endpoints, loopback health probes.
   4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch found no usable file for chopratejas/kompress-v2-base
   4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch refused for chopratejas/kompress-v2-base (HEADROOM_OFFLINE is set): HEADROOM_OFFLINE is set: refusing outbound network access for HuggingFace download of chopratejas/kompress-v2-base/onnx/kompress-int8-wo.onnx to huggingface.co. Unset HEADROOM_OFFLINE, or turn off the feature that needs this connection.
(smoke dir: $SMOKE)

$ HOME=$T/home HEADROOM_WORKSPACE_DIR=$T/ws CODEX_HOME=$T/codex python -c "<compact_lossless with collapse_runs patched to raise RuntimeError('token sk-live-123 in tool output')>"
headroom.transforms.lossless_compaction - WARNING - Lossless compaction (log) failed (RuntimeError) on 205 chars; leaving content as-is
unchanged: True

Runtime Rollout Safety

• Rollout-managed feature(s): None.
• Minimum rollout channel: Not applicable; no rollout-managed behaviour is touched.
• Stable/default behavior changed: Only log output. Five failure paths now log at WARNING; one of them was DEBUG.
• Kill switch / disable path: Not applicable; the change only adds log lines.
• Unsafe override required: No.
• Qualification impact: None.
• Rollback path: Revert this single commit.

Review Readiness

• I have performed a self-review
• This PR is ready for human review

Checklist

• My code follows the project's style guidelines
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I did not edit CHANGELOG.md — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this)

Screenshots (if applicable)

Not applicable.

Additional Notes

Part of maintainer-requested cleanup #4136 (dead code, duplication, debuggability), split into small PRs for review.

Documentation changes are not in this PR. They are recorded in the cleanup docs ledger for the single follow-up docs PR.

The image tile-optimizer item from the audit was dropped. A non-decodable image makes PIL raise UnidentifiedImageError there, so a warning would fire on ordinary bad input. There is no once-guard. Each warning fires only when a compressor raises or the CCR store fails, the same idiom as the existing CCR write-through warnings.

🤖 Generated with Claude Code

[github-actions]

PR governance

This PR follows the template and is marked ready for human review.

[JerrettDavis]

Reviewed head f255772, including the comments and your replies. The warning sanitization and failure-path coverage are useful improvements, but moving arbitrary exception details to DEBUG leaves a privacy issue in all five changed paths.

I invoked the real lossless-compaction and Kompress CCR-store entry points with injected failures echoing the original tool output. Both correctly preserved their fallback behavior, and WARNING omitted the canary. Formatting the DEBUG records, however, exposed the tool-output canary through exc_info=True. An opt-in diagnostic level can still put private prompts, source content or credentials into retained/shared logs.

Please keep failure diagnostics payload-safe at every enabled level across batch compression, cross-turn dedup, lossless compaction, Kompress storage and SmartCrusher mirroring. Preserve exception class/count/hash metadata and existing fallback behavior; exclude arbitrary exception messages/chains that can echo content. Extend the existing five-path regression to format captured DEBUG and WARNING records and assert content/credential canaries are absent from every record. Checking WARNING alone misses the rendered traceback leak.

All 32 selected failure-logging/lossless tests passed (exit 0). The independent two-path formatted-log reproduction passed its assertions (exit 0). Main a9815bb is an ancestor, diff check passes, and GitHub reports conflict-free. Qualification stops at this concrete diagnostic issue. No contributor source edits or merge performed.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 95.78059% with 10 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
headroom/log_safety.py	95.07%	6 Missing and 4 partials ⚠️

📢 Thoughts on this report? Let us know!