Repository navigation
Show own-run analysis scope beside durable finding history #301
Copy link
Copy link
Open
Labels
enhancementNew feature or requestNew feature or requestpriority: P2Do nextDo nextready-for-agentFully specified, ready for an AFK agentFully specified, ready for an AFK agent
Description
Parent
#95. Specification: #295 — Spec: per-run analysis scope and honest finding trends.
What to build
The standing quality report displays its own full Layer 2 scan scope beside
findings and retains one history entry per run/attempt. Reduced scope and
changed checking conditions are visible without opening an artifact. Failed
or incomplete attempts are not recorded as zero findings, and legacy history
is retained without inventing scope measurements.
This step depends on Layer 2/Semgrep evidence, not on language-job or Rust
instrumentation. The report does not import results from a quality run.
Acceptance criteria
- Current findings and scope refer to the same full Layer 2 execution. The issue shows per-tool measurements with units, gaps, unavailable values and bounded diagnostic excerpts linked to complete evidence.
- Ordinary scanner failure or missing success-handoff data can still produce a failed/incomplete attempt entry and artifact. Preserve existing operational failure signaling without making scope collection a new gate.
- New history identity includes run and attempt; distinct same-day runs/attempts remain distinct, while repeated processing of the same identity is idempotent.
- Keep the latest 52 entries in deterministic execution order. Legacy daily rows remain as legacy entries with unknown scope. Explain that retention counts entries, not days.
- Persist compact comparison metadata in the standing issue so comparisons do not require old artifacts to remain downloadable.
- Compare with the immediate chronological predecessor in the report stream. Legacy, incomplete or unavailable predecessor data cannot become a zero baseline or be silently skipped to hide an outage.
- Finding deltas appear beside meaningful per-tool scope deltas. Smaller examined scope, more analysis gaps or more observed policy exclusions are visibly qualified; fewer findings are not automatically called an improvement.
- Changed tool selection, versions, active rules, effective configuration/exclusions or scan mode are flagged even if counts are equal. Meaningful deltas may remain visible with qualification; incompatible units/methods are not numerically compared.
- Missing comparison metadata prevents claims of stable conditions. Counts alone do not claim identical file/rule identity or diagnose why the scope changed.
- Malformed optional scope metadata preserves parseable previous history and marks comparison unavailable. Unrecoverable prior history is reported without silently replacing it with an empty/fresh history.
- Protect the single issue's read/modify/write cycle against concurrent updates and out-of-order completion. An older run cannot erase newer entries or become the current report by finishing later.
- Renderer/action fixtures cover same-day attempts, idempotency, out-of-order arrival, concurrent-update handling, legacy rows, the retention cap, malformed history and multiline errors. Workflow wiring checks cover error-path publication and same-run scope handoff.
- Report-only runs never claim Layer 1 or Rust execution. No cross-run artifact lookup, central storage, Consumer changes, live observations or releases are included.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or requestpriority: P2Do nextDo nextready-for-agentFully specified, ready for an AFK agentFully specified, ready for an AFK agent
Activity