Skip to content

Show own-run analysis scope beside durable finding history #301

Open

Description

@maximalcode

Parent

#95. Specification: #295 — Spec: per-run analysis scope and honest finding trends.

What to build

The standing quality report displays its own full Layer 2 scan scope beside
findings and retains one history entry per run/attempt. Reduced scope and
changed checking conditions are visible without opening an artifact. Failed
or incomplete attempts are not recorded as zero findings, and legacy history
is retained without inventing scope measurements.

This step depends on Layer 2/Semgrep evidence, not on language-job or Rust
instrumentation. The report does not import results from a quality run.

Acceptance criteria

  • Current findings and scope refer to the same full Layer 2 execution. The issue shows per-tool measurements with units, gaps, unavailable values and bounded diagnostic excerpts linked to complete evidence.
  • Ordinary scanner failure or missing success-handoff data can still produce a failed/incomplete attempt entry and artifact. Preserve existing operational failure signaling without making scope collection a new gate.
  • New history identity includes run and attempt; distinct same-day runs/attempts remain distinct, while repeated processing of the same identity is idempotent.
  • Keep the latest 52 entries in deterministic execution order. Legacy daily rows remain as legacy entries with unknown scope. Explain that retention counts entries, not days.
  • Persist compact comparison metadata in the standing issue so comparisons do not require old artifacts to remain downloadable.
  • Compare with the immediate chronological predecessor in the report stream. Legacy, incomplete or unavailable predecessor data cannot become a zero baseline or be silently skipped to hide an outage.
  • Finding deltas appear beside meaningful per-tool scope deltas. Smaller examined scope, more analysis gaps or more observed policy exclusions are visibly qualified; fewer findings are not automatically called an improvement.
  • Changed tool selection, versions, active rules, effective configuration/exclusions or scan mode are flagged even if counts are equal. Meaningful deltas may remain visible with qualification; incompatible units/methods are not numerically compared.
  • Missing comparison metadata prevents claims of stable conditions. Counts alone do not claim identical file/rule identity or diagnose why the scope changed.
  • Malformed optional scope metadata preserves parseable previous history and marks comparison unavailable. Unrecoverable prior history is reported without silently replacing it with an empty/fresh history.
  • Protect the single issue's read/modify/write cycle against concurrent updates and out-of-order completion. An older run cannot erase newer entries or become the current report by finishing later.
  • Renderer/action fixtures cover same-day attempts, idempotency, out-of-order arrival, concurrent-update handling, legacy rows, the retention cap, malformed history and multiline errors. Workflow wiring checks cover error-path publication and same-run scope handoff.
  • Report-only runs never claim Layer 1 or Rust execution. No cross-run artifact lookup, central storage, Consumer changes, live observations or releases are included.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions