Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions .github/skills/pr-triage/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,11 @@ action must refuse — even if `statusCheckRollup.state` reports
bot checks (`Mergeable`, `WIP`, `DCO`, `boring-cyborg`) while
`Tests`, `CodeQL`, and newsfragment-check sit in
`action_required`; trusting the rollup there fills the
maintainer-review queue with PRs whose real CI never ran.
maintainer-review queue with PRs whose real CI never ran. The
guard applies identically to the
[`mark-ready-with-ping`](actions.md#mark-ready-with-ping)
action — any code path that adds the `ready for maintainer
review` label runs the REST check first.
Implementation recipe: [`actions.md#mark-ready`](actions.md).

**Golden rule 2 — propose in groups, fall back to per-PR.** The
Expand Down Expand Up @@ -326,6 +330,7 @@ string using [`suggested-actions.md`](suggested-actions.md):
| `deterministic_flag` — ≤2 CI failures, no conflicts, no threads, branch up-to-date | `rerun` |
| `deterministic_flag` — failures all match recent main-branch failures | `rerun` |
| `deterministic_flag` — static-check-only failures | `comment` |
| `deterministic_flag` — unresolved threads only, CI green, threads look addressed (heuristic) | `mark-ready-with-ping` (label + ping reviewers to confirm) |
| `deterministic_flag` — unresolved review threads only, CI green | `ping` |
| `deterministic_flag` — author has >3 flagged PRs | `close` |
| `deterministic_flag` — other | `draft` |
Expand All @@ -351,10 +356,13 @@ the order:
2. `deterministic_flag` with action `close` — destructive,
review individually
3. `deterministic_flag` with actions `draft` / `comment` /
`rebase` / `rerun` — in that order
`rebase` / `rerun` / `ping` — in that order
4. `stale_review` → `ping`
5. `passing` → `mark-ready`
6. Stale sweeps (`stale_draft` → `close`, `inactive_open` →
5. `deterministic_flag` → `mark-ready-with-ping` (label-bearing
group, presented just before plain `mark-ready` so the
maintainer reviews all label-add proposals back-to-back)
6. `passing` → `mark-ready`
7. Stale sweeps (`stale_draft` → `close`, `inactive_open` →
`draft`, `stale_workflow_approval` → `draft`)

For each group, present one screen worth of headline info
Expand Down
76 changes: 76 additions & 0 deletions .github/skills/pr-triage/actions.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,81 @@ error; this is the only action of the skill whose sole purpose

---

## `mark-ready-with-ping` — promote a likely-addressed PR + ping reviewers

A composite of `mark-ready` plus a `ping` comment. Used when
the only `deterministic_flag` signal is unresolved review
threads **and** the
[`unresolved_threads_only_likely_addressed`](classify.md#sub-flag-unresolved_threads_only_likely_addressed)
sub-flag is true (the author has engaged with every unresolved
thread via a post-comment commit or an in-thread reply).

**Same mandatory pre-mutation guard as `mark-ready`.** Run
the `action_required` REST check first and refuse — by
reclassifying as `pending_workflow_approval` — if any workflow
run is awaiting approval. The reasoning in the
[`mark-ready`](#mark-ready--add-ready-for-maintainer-review-label)
section above applies identically here.

Order of operations: **post the comment first, then add the
label.** The comment names the reviewers and asks them to
re-look at the threads; the label then routes the PR into the
review queue. If the label is added first, the reviewers see a
"ready for maintainer review" notification before the comment
that explains *why* it was promoted, which reads as the bot
mark-ready'ing without context.

```bash
# 1. Pre-check: refuse if any workflow run is awaiting approval (same as mark-ready).
head_sha=$(gh api "repos/<owner>/<repo>/pulls/<N>" --jq '.head.sha')
pending=$(gh api "repos/<owner>/<repo>/actions/runs?head_sha=${head_sha}&status=action_required&per_page=10" \
--jq '.workflow_runs | length')
if [ "$pending" -gt 0 ]; then
echo "refuse mark-ready-with-ping: <N> has ${pending} workflow run(s) awaiting approval at ${head_sha}" >&2
# Reclassify: this PR is really pending_workflow_approval.
exit 2
fi

# 2. Post the ping comment (mark-ready-with-ping template).
gh pr comment <N> --repo <repo> --body-file /tmp/pr-<N>-mark-ready-with-ping.md

# 3. Apply the label.
gh pr edit <N> --repo <repo> --add-label "ready for maintainer review"
```

Body template:
[`comment-templates.md#mark-ready-with-ping`](comment-templates.md).

The body must `@`-mention every reviewer whose unresolved
thread we believe was addressed (so they get the notification)
and `@`-mention the PR author once at the top (so the
contributor sees the rationale). The list of reviewers comes
from the unresolved-thread reviewers captured during
classification.

If the label step fails after the comment is already posted,
**do not retry the comment** — surface the label-add error
to the maintainer and let them apply the label manually. A
duplicate ping comment is the worse of the two failure modes.

If the comment step fails (network blip, rate-limit), do not
proceed to the label — the maintainer would then see a PR
labelled `ready for maintainer review` with no explanation of
how it got there.

### Falling back to plain `ping`

If the post-confirmation drill-in (e.g. the maintainer pulled
the PR out of the group with `[P]ick`) reveals that the
threads are *not* actually addressed, the maintainer can
override the action to `ping`. The override drops the label
step entirely and posts the regular
[`reviewer-ping`](comment-templates.md#reviewer-ping) body
instead. See
[`interaction-loop.md#group-action-override`](interaction-loop.md).

---

## `rerun` — rerun failed CI workflow runs

Multi-step. We need to find the workflow runs for this PR's
Expand Down Expand Up @@ -381,6 +456,7 @@ For every action that includes a comment, post the comment
| `close` | post comment → close → label |
| `flag-suspicious` | post comment → close → label *(per PR in the batch)* |
| `mark-ready` | label only |
| `mark-ready-with-ping` | post comment → label |
| `rerun` | rerun (no comment) |
| `rebase` | update-branch (no comment) |
| `ping` | post comment |
Expand Down
35 changes: 34 additions & 1 deletion .github/skills/pr-triage/classify.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,39 @@ can't be reviewed yet. The exact *action* to propose depends on
static-check failures → comment, etc.) — see
[`suggested-actions.md`](suggested-actions.md).

#### Sub-flag: `unresolved_threads_only_likely_addressed`

A boolean computed alongside the C2 classification. It is
`true` when **all** of the following hold:

- The only `deterministic_flag` signal that fired for this PR is
unresolved review threads (no `CONFLICTING` mergeable state,
no out-of-grace CI failure).
- Every unresolved thread has post-first-comment activity by
the PR author — **either** an author reply later in the same
thread (`reviewThreads.nodes.comments.nodes` has a node where
`author.login == pr.author.login` and `createdAt >` the first
comment's `createdAt`), **or** a commit was pushed after the
thread's first-comment `createdAt`
(`commits(last:1).committedDate > comment.createdAt`).
- The PR's latest commit `committedDate` is **after** the most
recent unresolved-thread first-comment `createdAt` — i.e. the
most recent author push is plausibly the resolution, not
predating the reviewer's feedback.

The flag is consumed only by `suggested-actions.md` to choose
between the existing `ping` action and the new
[`mark-ready-with-ping`](actions.md#mark-ready-with-ping)
action — it does **not** create a separate classification.

**Heuristic, not authoritative.** The maintainer still confirms
the proposed action and the comment we post invites the
reviewer to push back if the threads aren't actually resolved
(see [`comment-templates.md#mark-ready-with-ping`](comment-templates.md)).
Conservative on purpose: a false-negative degrades to the
existing `ping` behaviour; a false-positive would advance a PR
that isn't actually ready, which is the worse failure mode.

### C2b. `stale_copilot_review`

**Condition.** The PR has at least one unresolved review thread
Expand Down Expand Up @@ -413,7 +446,7 @@ relies on:
| Pre-filter 5 (active maintainer conversation) | `comments(last:10).nodes.{author.login,authorAssociation,bodyText,createdAt}`, `latestReviews.nodes.{author.login,submittedAt}`, `commits(last:1).nodes.commit.committedDate` |
| `pending_workflow_approval` | `statusCheckRollup.state`, `authorAssociation`, `head_sha` |
| `stale_copilot_review` | `reviewThreads.nodes.{isResolved,comments.nodes.{author.login,createdAt,url}}`, `comments(last:10).nodes.{author.login,createdAt}` (to detect author replies after the Copilot comment) |
| `deterministic_flag` | `mergeable`, `statusCheckRollup.{state,contexts}`, `reviewThreads.nodes.{isResolved,comments.nodes.authorAssociation}`, `updatedAt`, `comments(last:10).nodes.{author.login,authorAssociation,createdAt}` |
| `deterministic_flag` | `mergeable`, `statusCheckRollup.{state,contexts}`, `reviewThreads.nodes.{isResolved,comments.nodes.{author.login,authorAssociation,createdAt}}`, `updatedAt`, `comments(last:10).nodes.{author.login,authorAssociation,createdAt}`, `commits(last:1).nodes.commit.committedDate`, `author.login` (for the `unresolved_threads_only_likely_addressed` sub-flag — needs author replies in-thread, hence `comments(first: 5)` per thread instead of `first: 1`, see [`fetch-and-batch.md`](fetch-and-batch.md)) |
| `stale_review` | `latestReviews.nodes.{state,author.login,submittedAt}`, `commits(last:1).nodes.commit.committedDate`, `comments(last:10)` |
| `already_triaged` | `comments(last:10).nodes.{author.login,bodyText,createdAt}`, viewer login, `commits(last:1).nodes.commit.committedDate` |
| `passing` | `statusCheckRollup.state`, `statusCheckRollup.contexts`, `mergeable`, `reviewThreads.totalCount` |
Expand Down
59 changes: 53 additions & 6 deletions .github/skills/pr-triage/comment-templates.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,12 +58,12 @@ Rules for the footer:

- **Always include it** on every contributor-facing comment the
skill posts — `draft`, `comment-only`, `close`,
`review-nudge`, `reviewer-ping`, `stale-draft-close`,
`inactive-to-draft`, `stale-workflow-approval`. The only
exception is the `suspicious-changes` template, which is
short, operationally sensitive, and already directs the
contributor to maintainers on Slack — adding the footer there
would dilute the signal.
`review-nudge`, `reviewer-ping`, `mark-ready-with-ping`,
`stale-draft-close`, `inactive-to-draft`,
`stale-workflow-approval`. The only exception is the
`suspicious-changes` template, which is short, operationally
sensitive, and already directs the contributor to maintainers
on Slack — adding the footer there would dilute the signal.
- **Do not paraphrase it.** Post the block verbatim. If the
wording needs to change, update this section and propagate —
do not drift per-template.
Expand Down Expand Up @@ -292,6 +292,53 @@ in-thread reply.

---

## Mark ready with ping

*(`mark-ready-with-ping` — promote-and-invite-reviewers comment)*

Used when the action is `mark-ready-with-ping` (see
[`actions.md#mark-ready-with-ping`](actions.md)). The PR's
only outstanding signal is unresolved review threads, the
[`unresolved_threads_only_likely_addressed`](classify.md#sub-flag-unresolved_threads_only_likely_addressed)
heuristic fired, and we are promoting the PR to
`ready for maintainer review` while inviting the original
reviewer(s) to confirm the resolution.

```markdown
@<author> — Your unresolved review thread(s) from <reviewers> appear to have been addressed (post-review commits and/or in-thread replies on every thread, with the latest commit pushed after the most recent thread). I've added the `ready for maintainer review` label so the PR re-enters the maintainer review queue.

<reviewers> — could you take another look when you have a chance? If you agree the feedback was addressed, please mark the threads as resolved so the queue signal stays accurate. If a thread still needs work, please reply in-line — @<author> will follow up.

<ai_attribution_footer>
```

Notes on the body:

- **`@<author>` is mentioned once at the top.** They get one
notification with the rationale (so they understand why the
label appeared) and a clear ask if a thread comes back open.
- **Every reviewer with an unresolved thread is `@`-mentioned
once** in the second paragraph. They get the prompt that
matters for them — "please re-look and resolve the threads if
you agree".
- **No "no rush" line.** Unlike the `draft` / `comment-only`
templates, this one is announcing forward motion (PR
promoted), not asking the author to fix something — the
decompression line would read as out of place.
- **No "thanks!"** — per the global tone rules, sign-offs are
noise.
- **The `<ai_attribution_footer>` still applies** so the
reviewer knows the promotion is AI-drafted; if the
heuristic was wrong they have a clear cue to push back
rather than assuming a maintainer made the call.

If the heuristic was wrong (the reviewer disagrees that the
threads are addressed), the reviewer can re-open a thread,
remove the label, or comment back — the PR re-enters the
unresolved-threads triage path on the next sweep.

---

## Stale draft close

*(`stale-draft-close` — stale draft closing comment)*
Expand Down
12 changes: 11 additions & 1 deletion .github/skills/pr-triage/fetch-and-batch.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,17 @@ query(
totalCount
nodes {
isResolved
comments(first: 1) {
# `first: 5` rather than `first: 1`: the
# `mark-ready-with-ping` heuristic in
# [`suggested-actions.md`](suggested-actions.md) needs
# to see whether the PR author has replied in-thread
# after the reviewer's first comment. 5 is the smallest
# window that catches the typical "reviewer comment →
# author reply" exchange without blowing GraphQL
# complexity (30 threads × 5 comments × 20 PRs = 3000
# nodes, well under the ceiling that `first: 10` here
# would push us toward).
comments(first: 5) {
nodes {
author { login }
authorAssociation
Expand Down
15 changes: 11 additions & 4 deletions .github/skills/pr-triage/interaction-loop.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,12 +47,17 @@ the groups in this fixed order:
8. `(deterministic_flag, ping)` — batchable (unresolved threads
from collaborators).
9. `(stale_review, ping)` — batchable.
10. `(passing, mark-ready)` — batchable.
11. `(stale_draft, close)` — batchable but with extra per-PR
10. `(deterministic_flag, mark-ready-with-ping)` — batchable
(unresolved threads that the heuristic believes are
addressed; presented just before the plain mark-ready group
because both end with the `ready for maintainer review`
label going on).
11. `(passing, mark-ready)` — batchable.
12. `(stale_draft, close)` — batchable but with extra per-PR
confirm inside the batch (these are rarely wrong but when
wrong they're very wrong).
12. `(inactive_open, draft)` — batchable.
13. `(stale_workflow_approval, draft)` — batchable.
13. `(inactive_open, draft)` — batchable.
14. `(stale_workflow_approval, draft)` — batchable.

The ordering is chosen so the maintainer always faces the
riskiest decisions first, while their attention is fresh. The
Expand Down Expand Up @@ -201,6 +206,7 @@ safe alternatives:
| `rebase` | `comment`, `skip` |
| `rerun` | `comment`, `skip` |
| `mark-ready` | `skip` |
| `mark-ready-with-ping` | `ping` (fall back to plain reviewer ping if the maintainer thinks the heuristic over-reached), `skip` |
| `ping` | `comment`, `skip` |
| `close` (deterministic_flag) | — (no overrides — use `[E]` to downgrade individually) |
| `close` (stale_draft) | `draft`, `skip` |
Expand Down Expand Up @@ -330,6 +336,7 @@ PRs acted on: 22
- rebased: 4
- reruns triggered: 3
- marked ready: 3
- marked ready w/ ping: 1
- pings posted: 2
PRs skipped: 15 (12 already triaged / inside grace, 2 bot, 1 collaborator)
PRs left pending: 10 (reached [Q] before classifying)
Expand Down
33 changes: 33 additions & 0 deletions .github/skills/pr-triage/suggested-actions.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ currently-flagged PRs by the same author seen **this page**.
| Only CI failures, *some* failures match main-branch failures | `rerun` | "K/N CI failures match recent main-branch PRs — likely systemic" |
| Only CI failures, every failed check is a static check (`ruff`, `mypy-*`, `pre-commit`, etc.) | `comment` | "Only static-check failures — deterministic, needs a code fix, not a rerun" |
| Only CI failures, `failed_count <= 2`, no conflicts, no unresolved threads, `commits_behind <= 50` | `rerun` | "N CI failure(s) on otherwise clean PR — likely flaky, suggest rerun" |
| Unresolved review threads only, CI green, no conflict, `unresolved_threads_only_likely_addressed` is true (see [`classify.md#sub-flag-unresolved_threads_only_likely_addressed`](classify.md)) | `mark-ready-with-ping` | "K unresolved thread(s) from <reviewers> appear addressed (post-review commits / in-thread author replies) — promote to ready-for-review and ping reviewers to confirm" |
| Unresolved review threads only, CI green, no conflict | `ping` | "K unresolved review thread(s) from <reviewers> — ping author + reviewers with the [`reviewer-ping`](comment-templates.md#reviewer-ping) template" |
| No CI at all (rollup empty / only bot contexts), `mergeable != CONFLICTING` | `rebase` | "No real CI checks triggered, branch mergeable — rebase to re-trigger" |
| *(fallback)* | `draft` | "Has quality issues — convert to draft with comment listing violations" |
Expand Down Expand Up @@ -201,6 +202,38 @@ colleague's PR to draft is an overreach.

---

## Choosing between `ping` and `mark-ready-with-ping`

Both actions handle the same upstream classification —
`deterministic_flag` whose only signal is unresolved review
threads. The difference is what we believe about the threads:

- `ping` — the threads are unresolved and **may not yet be
addressed**; the safe default is to nudge the author to
address each one (or explain why the feedback doesn't apply)
and mark them resolved themselves.
- `mark-ready-with-ping` — the threads are unresolved but
the author has already engaged with each one (post-comment
push or in-thread reply, plus the latest commit post-dates
the most recent unresolved thread). The PR is plausibly
ready for the maintainer to take another look; we promote
it with the `ready for maintainer review` label *and* ping
the original reviewer(s) to confirm and resolve their
threads. See [`actions.md#mark-ready-with-ping`](actions.md)
for the mutation, [`comment-templates.md#mark-ready-with-ping`](comment-templates.md)
for the body, and [`classify.md#sub-flag-unresolved_threads_only_likely_addressed`](classify.md)
for the heuristic.

The heuristic is conservative — when in doubt it returns
`false` and the PR falls through to plain `ping`. The
maintainer still confirms the action; the comment we post
explicitly invites the reviewer to push back if a thread is
not actually addressed, so a heuristic false-positive
self-corrects on the next round-trip rather than silently
landing the PR.

---

## Overriding at group level

The interaction loop lets the maintainer override the suggested
Expand Down
Loading