Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions openspec/specs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,20 @@ Spec-driven backlog. Start at **[backlog.md](backlog.md)** — the phase → epi

Authoring rules: [../config.yaml](../config.yaml). Levels: L0–L3 per
[ADR-0006](../../docs/adr/0006-testing-strategy.md); design artifacts use `Level: doc`.

## Gate-wiring rule for guards (applies to every hygiene epic: CIH, NIT, TDS, TCC, DEP, CMY)

A gate that cannot block a merge is not a gate. Every new guard script, **in every mode it has**
(`--self-test`, `--classify`, `--transitions`, ...), must (1) run on `pull_request` in the
required `verify` job (or in a job that is in the `needs` of the required aggregator), as its
**own argument-pinned step** in the style of `verify.yaml:63-69` (unargumented or fully
argumented, no `if:`, no `continue-on-error:`, asserted by `workflow_pins_test.sh`), and (2) run
locally via `task check`, with each mode pinned in `STAGE_BODY_PINS`
(`hack/audit/exitgate_test.sh:207`). Reaching CI only through `release-exitgate` (push-only,
`verify.yaml:213-214`) is not enough: an unwired mode is the D-159 orphan class. Each epic's
Exit section references this rule instead of restating it.

**Post-merge evidence.** A REQ whose evidence cannot exist before the merge (a `workflow_run` or
scheduled workflow that only fires from the default branch, a bot that reads its config from
`main`, "N days green") is marked `post-merge` and links an open follow-up issue that owns it.
The story is not done until that issue closes with the evidence.
34 changes: 34 additions & 0 deletions openspec/specs/backlog.md
Original file line number Diff line number Diff line change
Expand Up @@ -718,6 +718,40 @@ S02 cannot start until the operator creates the bestpractices.dev project (INBOX
| SEC-SC-S01 | Native Go fuzz targets on YAML/JSON/HCL differ (+ CI smoke) | **[autonomous]** | none | **do first** — Scorecard Fuzzing (#3); untrusted-byte crash/fail-open fence |
| SEC-SC-S02 | OpenSSF Best Practices passing badge + honest evidence page | **[operator-gated]** | operator creates the bestpractices.dev project | Scorecard CII-Best-Practices (#6); no fake README badge |

## Phase 5 — OSS quality-signal epics (2026-10-05 comparison; SPECIFIED, NOT STARTED)

Cross-cutting hygiene epics from a comparison against the OSS reference project attune
(github.com/attune-io/attune). Every new guard follows the **gate-wiring rule** in
[README.md](README.md). Landing order (one story = one issue = one PR) is in the table.

| Order | Story | Epic spec | Status |
| --- | --- | --- | --- |
| 1 | CIH-S00 restore green main (AUD-S18 determinism reports 0 runs) | [CIH](p5-cih-ci-hardening/spec.md) | ready |
| 2 | CIH-S01 `concurrency` (PR-number key) | CIH | ready |
| 3 | CIH-S02 zizmor gate + fix `release.yaml` cache-poisoning | CIH | ready |
| 4 | CIH-S03 dependency-review (+ operator: make required) | CIH | ready / operator for required check |
| 5 | NIT-S02 CI-failure reporter (`workflow_run`) | [NIT](p5-nit-nightly-integration/spec.md) | ready after CIH-S02 |
| 6 | TCC-S01 go.mod bump + `go-version-file` + tool check (one PR) | [TCC](p5-tcc-toolchain-consistency/spec.md) | ready after CIH-S00 |
| 7 | TCC-S02 pin baseline (`GOVULNCHECK_VERSION`, floors) | TCC | after TCC-S01 |
| 8 | TCC-S03 `mise.toml` + drift test | TCC | after TCC-S02 |
| 9 | TCC-S04 entry points (`check` = full local gate; no `verify` subset) | TCC | ready |
| 10 | TCC-S06 Go patch-lag sensor | TCC | blocked by NIT-S02 |
| 11 | CMY-S01 CONTRIBUTING + issue forms + agent pointers | [CMY](p5-cmy-community-agent-contract/spec.md) | ready |
| 12 | CMY-S02 evidence rules in GUIDELINES | CMY | ready |
| 13 | DEP-S01 Dependabot `cooldown` | [DEP](p5-dep-dependency-automation/spec.md) | ready |
| 14 | DEP-S03 App token / secrets / settings | DEP | **operator** (after required-check gates) |
| 15 | DEP-S04 contract doc + Sonar precondition | DEP | ready / operator precondition |
| 16 | DEP-S02 scheduled all-checks-green rebase merger | DEP | blocked by DEP-S03, CIH-S03, CIH-S00 |
| 17 | TDS-S01 fuzz runner + flake classifier | [TDS](p5-tds-test-depth/spec.md) | blocked by SEC-SC-S01 |
| 18 | TDS-S03 benchmarks (`workflow_dispatch`) | TDS | ready |
| 19 | NIT-S01 scheduled real e2e + no-silent-skip | NIT | blocked by E7-S07/P4-E1-S10, NIT-S02, operator infra |
| 20 | TCC-S05 dependency bot A/B/C | TCC | **operator decision** |
| 21 | TDS-S02 report-only mutation | TDS | deferred until NIT-S02 shipped |
| — | CIH-S04 `ci-gate` aggregate | CIH | deferred (trigger in spec) |
| — | harden-runner | CIH | deferred (needs block-mode egress allowlist plan) |

Cut: PR labeler/size labels, live branch-protection probe, `task verify` fast subset.

## Phase 5 — RVW six-review remediation (2026-09-25)

Full INVEST stories in [p5-rvw-review-remediation/spec.md](p5-rvw-review-remediation/spec.md).
Expand Down
166 changes: 166 additions & 0 deletions openspec/specs/p5-cih-ci-hardening/spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
# P5-CIH — CI hardening (green main, concurrency, workflow static analysis, dependency review)

**Epic ID / REQ prefix:** `CIH` / `REQ-CIH-Snn-nn`. Cross-cutting hygiene epic (same class as
`AUD` / `SEC-SC`). Does **not** consume E12–E14. Gate wiring: see the
[gate-wiring rule](../README.md) (D1) — every new guard and mode runs in the required `verify`
job and in `task check`.

**Problem.** Verified 2026-10-05: `verify.yaml` has two jobs (`verify` L38, `release-exitgate`
L213, push/schedule only) and **no `concurrency:`**; there is no workflow security linter beyond
`actionlint` and the pin tests in `hack/lint/workflow_pins_test.sh`; no `dependency-review-action`
on PRs; and **main is red at `1c4e71e`**: step "AUD audit exit gate (AUD-S18)" reports
`TestDeterminismDoubleRun passed 0 time(s) under -count=2` — a guard only push builds run, so no
PR could have caught it. Prior art: attune (`github.com/attune-io/attune`) `ci.yaml`.

**Pin-test convention.** `workflow_pins_test.sh` has **no argument handling**; its mutation
controls run inline on every plain invocation (already dozens — re-count at implementation and
assert the count only grows, never a literal). Extend it by adding controls; verify with
`bash hack/lint/workflow_pins_test.sh`. `--self-test` exists only on scripts an epic creates.

**Counterpoint (decided).** attune drops CI on push to `main` (squash-merge). assent **keeps**
push-to-main `verify`: `hack/release/verify-tag-gate.sh` (REQ-AUD-S03-01) needs a green
`verify.yaml` run on the tag SHA and merges are rebase-merge. Forbidden outcome (REQ-CIH-S03-03).

**Not in scope:** SHA-pin policy (exists); CodeQL/Scorecard changes; branch-protection edits other
than the `[operator]` steps named below.

**Deferred (not built here).** `step-security/harden-runner`: audit mode gates nothing, adds a
privileged third-party agent to token-holding jobs, and nobody reads the telemetry. Revive only
with a written **block-mode egress allowlist plan**. All harden-runner REQs were removed.

---

## CIH-S00 — Restore green main: AUD-S18 determinism guard reports 0 runs `[autonomous]`

**Depends on:** none. **First: every later epic's exit needs a green push `verify` on main.**

- **Root cause (verified):** `TestDeterminismDoubleRun` no longer exists — refactor `5a01ba0`
("single guarded engine entry Decide") deleted it; the name survives only in the `-run` regex at
`Taskfile.yml:253` and `verify.yaml:125`, so the gate runs 0 tests and AUD-S18 correctly reds.
- **Decision:** **restore an equivalent double-run determinism test over `Decide`** (same name, so
the existing `-run` regex and gate stay unchanged). Removing the name from the gate would be a
loosening and is not allowed. Given main, when the gate runs, then the restored test executes
under `-count=2` and the gate is green.
- Given a PR, when `verify` runs, then the determinism double-run evidence check also runs on the
PR path (argument-pinned step), so the class "guard only push builds run" cannot recur.
- **Adversarial** — a change that makes `TestDeterminismDoubleRun` (or any `-run` name in the
determinism gate) match zero tests is red on the PR: a pinned `verify` step asserts every `-run`
name in `Taskfile.yml` / `verify.yaml` determinism gates resolves to >= 1 existing test
(`go test -list`), so a rename/delete reds on the PR, not on push.

Requirements:

- **REQ-CIH-S00-01** — the double-run test executes twice. Test:
`hack/audit/exitgate_test.sh`; Verify:
`go test -count=2 -run '^TestDeterminismDoubleRun$' -v ./... | rg -c '^--- PASS: TestDeterminismDoubleRun'`
prints `2` (it fails today: the test does not exist); Level: L0
- **REQ-CIH-S00-02** *(adversarial)* — a `-run` name in the determinism gate that matches no
test is red on the PR, proven failing-direction on a fixture. Test:
`hack/lint/run_names_resolve_test.sh` (create); Verify:
`bash hack/lint/run_names_resolve_test.sh && bash hack/lint/run_names_resolve_test.sh --self-test`; Level: L0
- **REQ-CIH-S00-03** — push `verify` on main is green on the fix commit. Test: GitHub run;
Verify: `gh run list -R PlatformRelay/assent -w verify -b main -L1 --json conclusion`; Level: L3 (**post-merge**, D14)

---

## CIH-S01 — `concurrency:` on `verify` `[autonomous]`

**Depends on:** none.

- Given a PR with a newer push, then the in-flight run for the same PR is cancelled; given a
push to `main`, a tag, a schedule or `workflow_dispatch`, then nothing is cancelled.
- Key (D5): `group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}`
(not `head_ref`: fork branches named alike would collide); `cancel-in-progress:
${{ github.event_name == 'pull_request' }}`.
- **Adversarial** — unconditional `cancel-in-progress: true` or a `head_ref` key is red (a
cancelled `main` run leaves the tag SHA without green `verify`).

Requirements:

- **REQ-CIH-S01-01** — block present with the D5 key and condition. Test:
`hack/lint/workflow_pins_test.sh`; Verify: `bash hack/lint/workflow_pins_test.sh`; Level: L0
- **REQ-CIH-S01-02** *(adversarial)* — new inline controls for unconditional cancel and
`head_ref` key are red on the mutated copy; plain run stays green and lists them. Test: same;
Verify: `bash hack/lint/workflow_pins_test.sh`; Level: L0

---

## CIH-S02 — `zizmor --offline` gate; existing findings fixed `[autonomous]`

**Depends on:** none.

- Given the workflows, when `zizmor --offline --min-severity <pinned> .github/workflows` runs
(one exact pinned version >= 1.30.1, single-sourced in the workflow `env:` block; installed
via `uvx zizmor==X` or pipx `--require-hashes`, **not** `go install`), then findings fail.
- Today offline zizmor reports **2 high `cache-poisoning`** in `release.yaml` (setup-go cache in
the tag-triggered publish path) and 1 informational `superfluous-actions`. These are
**fixed**, not suppressed (`cache: false` on those `setup-go` steps; drop the superfluous
action). Any suppression in `.github/zizmor.yml` carries a `# why:` line; no blanket ignores.
- **No skip switch** (`ZIZMOR_SKIP*`/equivalent) may appear in any workflow (guarded).
- Later changes that add workflows (e.g. NIT-S02's `workflow_run` reporter -> `dangerous-triggers`)
must pass this gate in their own tasks.
- Fixture seeds a **third-party** mutable-ref action (`actions/*` ref pins are allowed by default).
- Local: `task check` requires `uvx` like it already requires `uv` + Python 3.12 (`Taskfile.yml`
comment ~L156); no silent skip.

Requirements:

- **REQ-CIH-S02-01** — gate step in `verify.yaml`, pinned version >= floor, `--min-severity` set,
no `if:`/`continue-on-error`. Test: `hack/lint/zizmor_gate_test.sh` (create); Verify:
`bash hack/lint/zizmor_gate_test.sh`; Level: L0
- **REQ-CIH-S02-02** — `release.yaml` cache-poisoning findings fixed with zero suppressions for
them. Test: same; Verify: `bash hack/lint/zizmor_gate_test.sh --self-test`; Level: L0
- **REQ-CIH-S02-03** *(adversarial)* — third-party mutable-ref fixture flagged; suppression
without `# why:` red; any skip env in a workflow red. Test: same; Verify:
`bash hack/lint/zizmor_gate_test.sh --self-test`; Level: L0

**Counterpoint.** Overlaps `actionlint` + pin tests on pinning; zizmor adds template injection,
permissions and `pull_request_target`/`workflow_run` audits.

---

## CIH-S03 — `dependency-review-action` on PRs; push-trigger fence `[autonomous · operator for required check]`

**Depends on:** none.

- Given a PR that adds/bumps a dependency, when the `pull_request`-only job runs, then
`actions/dependency-review-action` (SHA-pinned) fails at the action's **default
`fail-on-severity` (low)**; loosening is allowed only after a measured noisy trial and with its
own `# why:` line.
- Licences: `allow-licenses` (Apache-2.0-compatible set) plus per-module
`allow-dependencies-licenses` each with a reason — **not** the deprecated `deny-licenses`
(action issue #938). Unknown licences are reported, not failed.
- Works on Dependabot PRs without secrets. `[operator]`: make it a **required** check.
- **Forbidden outcome** — `verify.yaml` losing `push.branches: [main]` or the weekly schedule.

Requirements:

- **REQ-CIH-S03-01** — PR-only job, pinned, no `continue-on-error`, `allow-licenses` present,
no `deny-licenses`. Test: `hack/lint/workflow_pins_test.sh`; Verify:
`bash hack/lint/workflow_pins_test.sh`; Level: L0
- **REQ-CIH-S03-02** *(adversarial)* — a severity override without `# why:` or the step dropped
is red (inline control). Test: same; Verify: `bash hack/lint/workflow_pins_test.sh`; Level: L0
- **REQ-CIH-S03-03** *(forbidden outcome)* — `on:` keeps `push.branches: [main]` and `schedule`
with a comment citing `verify-tag-gate.sh`. Test: same; Verify:
`bash hack/lint/workflow_pins_test.sh`; Level: L0

---

## CIH-S04 — `ci-gate` aggregate job `[deferred]`

Path-filtered workflows already exist (`docs.yaml`, `schemas.yml`, `actionlint.yaml`,
`release.yaml` use `paths:`), but `ci-gate` **cannot aggregate other workflows**, and
CodeQL stays a separate required check. Bot auto-merge safety is handled directly in DEP-S02
(all check runs on the head SHA + green main), so this story stays deferred. **Trigger:** a new
PR-only job inside `verify.yaml` that needs aggregating. If built: `if: always()`, graded on
`contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled')` (never
`failure()`), every sibling job in `needs`, `skipped` = pass, required beside `verify` and CodeQL.
Requirements are written then.

---

## Exit

S00–S03 green on a push `verify` at main; wiring per the [gate-wiring rule](../README.md): every
new script/mode is a pinned `verify` step and a `task check` stage with `STAGE_BODY_PINS`
entries. `release-exitgate` is a **job** in `verify.yaml`, not a workflow.
77 changes: 77 additions & 0 deletions openspec/specs/p5-cmy-community-agent-contract/spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# P5-CMY — Community files and agent-contract hygiene

**Epic ID / REQ prefix:** `CMY` / `REQ-CMY-Snn-nn`. Cross-cutting hygiene epic. Keep small. Gate
wiring: see the [gate-wiring rule](../README.md) (D1).

**Problem.** Verified 2026-10-05: no `CONTRIBUTING.md`; `.github/ISSUE_TEMPLATE/` has two
Markdown templates and no `config.yml`; `CLAUDE.md` is a one-line `@AGENTS.md` import but there
is no `.github/copilot-instructions.md`; the verification discipline learned in review rounds is
not in `GUIDELINES.md`. Prior art: attune `CONTRIBUTING.md` and issue forms.

**Not in scope:** CODE_OF_CONDUCT/SECURITY (exist); PR labeler and size labels (cut: for a solo
maintainer they add a privileged workflow to protect for little reading value); Discussions; bots.

---

## CMY-S01 — Community files and agent pointers `[autonomous]`

**Depends on:** none. One PR.

- **CONTRIBUTING.md** states, linking to `AGENTS.md`/`GUIDELINES.md` rather than copying them,
these **seven** topics: the `:gitmoji: type(scope): summary` ASCII-shortcode convention (no
Unicode emoji); one logical change per commit; `task check` before pushing; TDD expectation;
**rebase-merge only** (no squash); the determinism and default-deny invariants (link); the
private vulnerability path (link to `SECURITY.md`). D-002: nothing employer/internal.
- **Issue forms:** `bug_report.yml` and `feature_request.yml` replace the `.md` templates
(required: version, forge, policy snippet, expected/actual); `config.yml` sets
`blank_issues_enabled: false` and routes vulnerabilities to the **private** advisory URL.
**Adversarial** — no field asks for secrets/tokens; no public-issue security route.
- **Agent pointers:** `CLAUDE.md` keeps its `@AGENTS.md` import and
`.github/copilot-instructions.md` is a one-line pointer to `AGENTS.md`. **Adversarial** — a
pointer with rule text or more than ~3 non-blank lines is red (a copy is a fork).

Requirements:

- **REQ-CMY-S01-01** — CONTRIBUTING exists with the seven topics and links. Test:
`hack/lint/community_files_test.sh` (create); Verify:
`bash hack/lint/community_files_test.sh`; Level: doc
- **REQ-CMY-S01-02** *(adversarial)* — forms parse with required fields, old `.md` removed,
security link private, no secret-asking field; pointers stay pointers. Test: same; Verify:
`bash hack/lint/community_files_test.sh --self-test`; Level: L0
- **REQ-CMY-S01-03** — passes sanitization. Test: `hack/check-sanitization.sh`; Verify:
`bash hack/check-sanitization.sh`; Level: doc

---

## CMY-S02 — Evidence and recurring-defect rules in GUIDELINES `[autonomous]`

Add four concise rules to `GUIDELINES.md` (Testing section) and one reference line from the
`specs:` rules in `openspec/config.yaml`:

1. **Evidence:** a verification row counts only if the named test visibly ran (`=== RUN` /
`-v` output names it); a `-run` filter that matches nothing is "ok" with no evidence.
2. **Controls:** a defect/mutation control must be seen to **go red on a real defect** and a
**no-op/equivalent change must leave it green**; a harness that reds on a no-op, or stays
green on a real defect, is broken, and proves nothing about the test under it.
3. **Flake/race:** a pass claimed for a flaky or concurrent behaviour needs `-count=N` (N stated,
`-race` where relevant); one green run is not evidence.
4. **Recurring defects:** a defect class seen twice gets its own automated sensor before the
third fix.

- **Counterpoint:** more rules, more reading; four bullets, each already enforced informally.
No token-matching test is added (a test that greps for heading words proves the words exist,
not that the rule is followed).

Requirements:

- **REQ-CMY-S02-01** — the four rules are present in `GUIDELINES.md`. Test: `GUIDELINES.md`;
Verify: `rg -c '^\*\*(Evidence|Controls|Flake/race|Recurring defects):' GUIDELINES.md` prints
`4`; Level: doc

---

## Exit

S01 and S02 landed; `task check` green. The S01 guard follows the
[gate-wiring rule](../README.md): pinned `verify` steps for each mode and a `task check` stage with
`STAGE_BODY_PINS` entries.
Loading
Loading