Skip to content

ci: scope pull request checks to what the diff can affect - #8946

Merged
waleedlatif1 merged 2 commits into
stagingfrom
ci/change-scoped-checks
Oct 12, 2026
Merged

waleedlatif1 merged 2 commits into
stagingfrom
ci/change-scoped-checks

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

Every PR ran the full suite. A library post with one .mdx and one cover .jpg (#8931) cost 107 job-minutes, about 755 vCPU-minutes, and took 21 minutes. Of the last 400 merged PRs, 75 were content-only, 7 desktop-only and 3 docs-app-only.

A new changes job (2 vCPU, ~10s) runs .github/scripts/ci-changes.sh. It replaces desktop-live-changes.sh and its job, fetches only the PR's base commit, and outputs which job groups the diff can affect.

Diff touches only Runs
Content: apps/sim/content/** except .tsx, and content-only public assets lint, test (og-image.test.ts checks covers), build (next build renders every post)
apps/docs/** lint, test (docs tests and two scripts tests read docs pages)
apps/desktop/** lint, test, desktop-live
Agent guidance (.agents, .claude, .cursor, AGENTS.md/CLAUDE.md, root README) lint (check:skills, check:guidance-refs)
Anything else, including any .github change, an empty diff or a diff that can't be computed everything

It fails safe:

  • The classifier only allows skips for paths on an allowlist. Anything unmatched, a failed fetch, or a missing base runs every job.
  • Push, dispatch and every other non-PR event always run the full suite, so the migrate → promote deploy chain never runs on partial checks.
  • packages/desktop-bridge and browser-protocol run everything, because 64 apps/sim files import them.
  • The ci aggregator accepts a skipped job only when changes set that job's flag to exactly false. A failed or skipped changes, or missing outputs, fails ci.
  • Lint always runs.

Effect: a content PR goes from about 755 to about 152 vCPU-minutes, and from 21 minutes of wall time to about 5.5.

Test plan

  • scripts/ci-changes.test.ts runs the real script against real git repos (20 cases, each guard checked by revert)
  • This PR touches .github, so its own run must show changes choosing the full suite and every job running

A `changes` job classifies the pull request's files against explicit
allowlists (.github/scripts/ci-changes.sh, replacing
desktop-live-changes.sh):

- content posts, author profiles and their public assets: lint, test, build
- apps/docs: lint, test
- apps/desktop: lint, test, desktop-live
- agent guidance (.agents, .claude, .cursor, AGENTS.md, CLAUDE.md, root README): lint

Any other file, any .github change, an empty diff, or a diff that cannot
be worked out runs everything; pushes and dispatches always run
everything. The live desktop suite keeps its own skip list, now including
the content asset folders it previously missed.

`ci` accepts a skip only when `changes` set that job's flag to exactly
`false`, so a failed or skipped `changes`, or a lost output, fails it.
@vercel

vercel Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Oct 12, 2026 10:21pm UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 6 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

Turn on auto-fix | Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[High impact] The PR appears safe to merge; no blocking issue remains.

Summary

This PR scopes pull request checks by changed paths while keeping full checks for non-PR events.

  • Pull requests now run only the checks their changed paths can affect.
  • The required ci check now accepts only classifier-approved skipped jobs.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Workflow event] --> B{Pull request?}
  B -->|Yes| C[Classify changed paths]
  B -->|No| D[Run full suite]
  C --> E[Run checks selected by flags]
  C -->|Unknown paths or diff failure| D
  E --> F[ci gate]
  D --> F
  L[Lint always runs] --> F
  F --> G{All checks passed or explicitly allowed skips?}
  G -->|Yes| H[Pass]
  G -->|No| I[Fail]
Loading

Reviews (2) · Last reviewed commit: "ci: move the ci gate into a tested scrip..." · Reviewed by Greptile

Comment thread .github/workflows/checks.yml Outdated
Comment thread scripts/ci-changes.test.ts
…ge flags

The ci aggregator's skip rules now live in .github/scripts/ci-gate.sh, read from
NEEDS, with table-driven tests covering every guard. The change-classifier test
helper fails on any flag value other than exactly true or false.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 8 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit d3e97f4 into staging Oct 12, 2026
48 checks passed
@waleedlatif1
waleedlatif1 deleted the ci/change-scoped-checks branch October 13, 2026 01:59

This branch was successfully deployed

1 active deployment
Preview — 6cd28a23 Deployed Oct 12, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant