Audit canary: three seeds per code domain, dealt across the domains

[nedtwigg]

The first canary measurement (run 37720586893) drew six application-security seeds out of six. The pool held twelve application-security seeds, one hosted seed and no supply-chain seed, so recall for the hosted and supply-chain domains was never measured. With this change every code domain has at least three seeds, and each draw takes an equal share from each domain.

Seeds added (inert .patch data under .github/audit/canaries/)

Each seed applies cleanly to main, alone and over the rest of the pool. Each removes or adjusts only the test case that would fail on it, so the seeded tree passes CI the way a regression on main would.

Domain	FAIL IF it breaks	Origin
hosted	security-hosted.md -> "Origin boundary" #2	reverses #904
hosted	security-hosted.md -> "Relay boundary" #4	synthetic
supply-chain	security-supply-chain.md -> "Bundled runtime" #2	synthetic
supply-chain	security-supply-chain.md -> "Disclosure" #2	synthetic
supply-chain	security-supply-chain.md -> "Disclosure" #4 (judgement item)	synthetic

Hosted now has 3 seeds (with the existing "Rendezvous boundary" #1 seed), supply-chain 3, and application-security 12.

What is not seeded. I left the supply-chain clauses that scripts/supply-chain-config.test.mjs judges unseeded: the pnpm and Renovate cooldowns, devEngines, the npx pin, the zizmor and actionlint pins, and release.yml's node-version-file. The domain is told to run that test as its evidence before any model reads the file, so a seed there would measure the test rather than the audit's recall. The harness has no deterministic-gate seed type, and none of these seeds needs one. Each supply-chain seed breaks a clause that no script answers. I checked that the existing hosted seed, and both new hosted seeds, pass scripts/e2e-lint.mjs.

Harness

• Sampling. shuffle now deals the pool round-robin across the three code domains: both the domains and each domain's seeds are ordered by run-id hash. A draw of CANARY_SEEDS=6 therefore takes two seeds from each domain. A uniform draw from 12/3/3 would still have left supply-chain out of about one run in four.
• Pool floor. The pool test requires at least three seeds per code domain. The new test fails against the old shuffle.
• Spec and rationale. security-audit.md -> "Canary recall" and its rationale record both changes. The caught/seeded figure now weights the domains equally.

Verification

• node --test scripts/security-audit-canary.test.mjs scripts/security-audit.test.mjs: 133 pass.
• pnpm lint:specs and pnpm lint:md pass. actionlint and workflow-lint pass.
• With all five new seeds applied: e2e-lint, outbound-lint, loopback-lint, workflow-lint and public-docs-lint pass; the generate-deps snapshots do not change; hosted tsc and boundary.test.ts pass.
• Not run locally: workers.test.ts, because Docker is unavailable here.

🤖 Generated with Claude Code

[cloudflare-workers-and-pages]

Deploying mouseterm with    Cloudflare Pages

Latest commit:	f975f89
Status:	✅  Deploy successful!
Preview URL:	https://06179220.mouseterm.pages.dev
Branch Preview URL:	https://audit-canary-seeds.mouseterm.pages.dev

View logs