Skip to content

feat(review): apply the review's own findings on the bot's PRs - #1203

Merged
max-sixty merged 6 commits into
mainfrom
review-fixes-own-prs
Sep 10, 2026
Merged

max-sixty merged 6 commits into
mainfrom
review-fixes-own-prs

Conversation

@max-sixty

@max-sixty max-sixty commented Sep 10, 2026 •

Copy link
Copy Markdown
Owner

When tend-review reviews a PR this bot authored, it posts its findings and a separate tend-mention session is dispatched to act on them as the author. That session starts cold — it has no memory of authoring — so it re-reads everything the reviewer already had in context, applies the fixes, and pushes, which fires another review. On max-sixty/leaf that hop costs ~$3.55 per boot at list price plus the re-review it triggers, for no independent judgment: it is the same model, told what to fix, starting over.

This makes the review session apply its own findings on PRs with no human author. Step 9 of the review skill already said that for third-party bot PRs (Dependabot, renovate); the bot's own PRs now follow the same rule, and the paragraph that told the reviewer to leave the work to an author session is gone. The order on a bot-authored PR: run the review, submit it against the head reviewed, resolve threads, then apply the findings in one batched push and poll CI to green. The review stays pinned to the head it reviewed — re-targeting onto the pushed head would make the queued synchronize run read it as already reviewed and exit without looking at the fix. Convergence is that run's job: it reviews the pushed head, resolves the threads the fix addressed, and fixes anything new, until a pass finds nothing. Human-authored PRs are unchanged: suggestions first, push only after the author accepts.

The mention verify script no longer dispatches on a review the bot wrote (fresh_inline and the bot-review-on-bot-PR branch are deleted; a human review on a bot PR still dispatches the author session). The dispatched prompt's "including when you authored the review" clause, the system prompt's self-loop exception, and the running-in-ci passages that narrated the old hand-off are cut over with it. Generated workflows in this repo are untouched per the pinned-release rule.

The gate's pre-existing mention-wins order stands: a bot review that names @<bot> still dispatches, and the weekly integration test's §6 relies on that to drive review → relay → dispatch → verify → handle with the single bot identity. The session it boots now exits silently per the self-loop guard, so §6 asserts the session ran (session-log artifact on the dispatch run, as §5 does for review) rather than a reply; outbound posting is already §4's assertion.

This was written by Claude Code on behalf of max-sixty

🤖 Generated with Claude Code

https://claude.ai/code/session_01BA3Qoj4SjTrvdPJp6eaNLf

A COMMENT review on a PR this bot authored used to dispatch a fresh
tend-mention session as the author. That session re-read from cold
everything the reviewer already had in context, applied the fixes the
review had just worked out, and pushed — measured at ~$3.55 a boot on
max-sixty/leaf. The reviewer now applies them itself, so step 9 has one
rule for PRs with no human author: third-party bot PRs and the bot's own.

Convergence still comes from the next session. The push fires
`synchronize`, and the queued run reviews the pushed head, resolves the
threads the fix addressed, and fixes whatever it finds new — which is why
the reviewer must leave its review pinned to the head it read rather than
re-target onto its own fix.

mention_verify no longer dispatches on a review the bot wrote; a human
review on a bot PR still boots the author session. A summons by name is
judged first, which is the order the weekly integration test drives the
review → dispatch → reply chain with, so it now has a test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BA3Qoj4SjTrvdPJp6eaNLf

@tend-agent tend-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mention_verify.py still dispatches a bot-authored review that names @<bot> — test_a_mention_wins_over_the_bot_review_gate locks that order — and the weekly integration test's §6 depends on it: it seeds @tend-agent … quoting this token verbatim: $NONCE from the bot account and asserts a reply carrying the nonce. The session that dispatch boots now reads two unconditional exits, though: the system prompt's self-loop guard and running-in-ci's Self-conversation Guard both lost the role-boundary carve-out that used to authorize it. What §6 asserts is the reply, not the dispatch, so it would fail as "mention path broken" with the gate working exactly as this PR intends. Suggestions inline keep the new exit rule and re-carve only the explicit summons.

Comment thread shared/system-prompt.md Resolved
§6 drove review → relay → dispatch → verify → handle with the bot's own
review naming itself and asserted a reply. The self-loop guard now has no
role-boundary exception, so the session that boots exits silently — the
reply would assert the behaviour the guard forbids. Assert the session-log
artifact on the dispatch run instead, as §5 does for review; outbound
posting is already §4's assertion.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BA3Qoj4SjTrvdPJp6eaNLf

@tend-agent tend-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

§6's new shape closes out last round's finding: the artifact assertion tests a session the guard now tells to exit silently, where the reply assertion tested behaviour the guard forbids. One inline suggestion below, plus two notes.

The registration cap now measures a different thing. Separate from the run-selection race open on this thread: before this change the seq 1 24 loop watched for any tend-mention run — the relay run, which registers within seconds of the review — so 120s was ample. --event repository_dispatch moves the target past a whole job, since the dispatch registers only once relay has been allocated a runner and has executed its POST. That hop has taken 8–31s on this repo with runners free and 8–12 minutes without (this pull_request_review run started at 07:56:19; its dispatch registered at 08:06:53). The fixture repo is idle, so the fast case should be the norm there — but a slow one exits "dispatch run never registered" and §8 files an issue for a path that was working. The scanning rewrite proposed on that thread absorbs this; a narrower fix to run selection would not.

Outside this diff. running-in-ci's "Investigating Other CI Runs" still says runs triggered by pull_request_review or pull_request_review_comment "sometimes produce no artifact when the session is very short". Under the relay those events only ever run relay — verify and handle skip, as they did on this PR's own review run — so they never produce one at all, and the session-bearing run is the repository_dispatch one §6 now keys on. Happy to push the correction.

Comment thread generator/tests/test_mention_verify.py Resolved
`relay` has no authorship gate, so §5's own review also yields a
`repository_dispatch` run — verify says no, handle skips, success with no
artifact — and it can register on either side of §6's snapshot. Latching
onto the newest dispatch run therefore fails on a working path. Scan every
dispatch run new since the snapshot and pass on the first with a
session-log artifact, keeping "never registered" and "no session ran"
apart. Correct running-in-ci's claim that review-event runs sometimes
produce no artifact: they only ever run `relay`; the session is on the
dispatch run.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BA3Qoj4SjTrvdPJp6eaNLf

@tend-agent tend-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The two failure messages don't stay separate the way §6's comment promises: REGISTERED is set inside the loop over $NEW, and the --jq narrows that to runs already at status == "completed". A dispatch run that registered and is still executing when the 60×10s budget expires therefore exits dispatch run never registered, and §8 files an issue here blaming the relay leg for a hop that fired — the slow week the split exists to describe. (The snippet I suggested last round carried the same filter; for the tracking to mean "registered" it has to sit on the unfiltered listing.) Dropping the status filter costs nothing: the loop only ever concludes positively per run, so an in-progress run is one extra artifact query that the next iteration re-scans, and an artifact that lands before its run completes is still proof a session ran. Suggestion inline.

…etes

`REGISTERED` was set only over runs already completed, so a dispatch run
still executing when the budget expired reported "never registered" and
blamed the relay leg for a hop that fired. List every new run; the
artifact is the proof a session ran, whatever the run's status.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BA3Qoj4SjTrvdPJp6eaNLf
tend-agent
tend-agent previously approved these changes Sep 10, 2026
One poll: list the dispatch runs created since the seeding review and
match the repo's session-log artifacts against that set in jq, instead of
snapshotting run ids, diffing with comm, and querying artifacts per run.
Seed the review through the API so the cutoff is GitHub's clock.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BA3Qoj4SjTrvdPJp6eaNLf

@tend-agent tend-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SEEDED is now the cutoff everything downstream keys on, but nothing asserts the seeding review was actually submitted. If the POST fails (a 422 on the body, a transient 5xx), SEEDED is empty and gh run list --created '>=' returns [] at exit 0 rather than erroring — so the loop burns its full 600s and exits dispatch run never registered, pointing §8 at the relay leg for a failure that happened before any dispatch could exist. That is the same wrong-diagnosis shape the two separate messages exist to prevent. Every other seed in the file is guarded the same way ([ -n "$PR" ] || { … }); suggestion inline.

@tend-agent
tend-agent dismissed their stale review September 10, 2026 21:35

Superseded by the review on a later commit.

An empty SEEDED made the poll burn its budget and blame the relay leg.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BA3Qoj4SjTrvdPJp6eaNLf
@max-sixty
max-sixty merged commit f6344b1 into main Sep 10, 2026
18 checks passed
@max-sixty
max-sixty deleted the review-fixes-own-prs branch September 10, 2026 23:46
@max-sixty max-sixty mentioned this pull request Sep 10, 2026
max-sixty added a commit that referenced this pull request Sep 10, 2026
Bumps the generator to 0.2.6, syncs the lockfile, and adds the `##
0.2.6` changelog section the release workflow publishes as the GitHub
Release notes.

Ships #1203 (the review session applies its own findings on PRs the bot
authored), #1199 (writable `TMPDIR` for sandboxed agents), #1201
(nightly bases its regeneration branch on the fetched default branch),
#1178, #1195, and #1196.

> _This was written by Claude Code on behalf of max-sixty_

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_016aR4DdFRBB5X4C1bYUGoGu

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
max-sixty added a commit that referenced this pull request Sep 11, 2026
Regenerated with `uvx tend@0.2.6 init` after the 0.2.6 release. The
generated workflows move to the 0.2.6 action pin and pick up #1203: the
mention verify script no longer dispatches a session on a review the bot
wrote, and the review prompt drops the clause that told that session to
action its own review.

`review-reviewers.yaml` is hand-maintained, so `init` leaves it on the
old pin. Restamped to 0.2.6 in the same commit, per `running-tend`'s
restamp step, which leaves every `max-sixty/tend/...` ref in
`.github/workflows/` at one version.

> _This was written by Claude Code on behalf of max-sixty_

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_016aR4DdFRBB5X4C1bYUGoGu

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
max-sixty pushed a commit to max-sixty/worktrunk that referenced this pull request Sep 11, 2026
Regenerates the `tend-*` workflows against tend 0.2.6 (from 0.2.5). The
workflow files themselves only move the `max-sixty/tend/claude@` pin and
the generated-by header; the adopter-visible changes ride in the action
and its bundled skills, plus one generated `tend-mention` condition.

**The agent sandbox gets a writable `TMPDIR`.** Root `/tmp` is read-only
for the agent, and until now bundled scripts and skill recipes wrote
there anyway — this repo carries a note in
`.claude/skills/running-tend/SKILL.md` telling sessions to route scratch
paths through `${TMPDIR:-/tmp}` for exactly that reason (#4063). 0.2.6
exports a standard writable `TMPDIR` under the sandbox user's home and
moves the bundled scripts onto it, so a recipe that reaches for a plain
`/tmp` path no longer dies on `Read-only file system`
([max-sixty/tend#1199](max-sixty/tend#1199)).

**A review the bot leaves on its own PR no longer summons a second
session.** `tend-mention`'s dispatch filter now returns early when the
review author is the bot, and `tend-review` applies its own findings on
PRs with no human author instead — the same rule step 9 of the review
skill already used for Dependabot and renovate PRs. Previously each
self-review round cost a cold `tend-mention` boot that re-read the whole
PR to apply fixes the reviewer had just written, then pushed and fired
another review
([max-sixty/tend#1203](max-sixty/tend#1203)).
This is the one change visible in the generated YAML — the
`tend-mention.yaml` diff below.

**The maintainer-tier check reads `authorAssociation` correctly.** The
shared skill block claimed `gh pr view --json` doesn't expose the field
at all; it does, nested on the comment and review objects, and only the
top-level key is rejected. Sessions deciding whether a requester may
direct a destructive action were being pushed to the REST API
unnecessarily
([max-sixty/tend#1178](max-sixty/tend#1178)).

Full comparison:
<max-sixty/tend@0.2.5...0.2.6>.

<details><summary>Generated <code>tend-mention.yaml</code> condition
change</summary>

The dispatch filter gains an early bail on a bot-authored review:

```python
# A review the bot wrote hands work to nobody: the review session
# applies the findings it raised. The mention checks run first, so
# naming the bot in a review still summons a session.
if review_author == bot:
    return verdict(False)
```

replacing the prior participation branch, which dispatched when the PR
author was also the bot and the review carried a body or fresh inline
comments. The prompt for a genuine `pull_request_review` dispatch drops
the clause that told the session to action a review it had authored
itself.

</details>

Co-authored-by: worktrunk-bot <254187624+worktrunk-bot@users.noreply.github.com>
max-sixty added a commit that referenced this pull request Sep 26, 2026
…s its findings (#1433)

#1203 moved the job of applying a bot-authored PR's findings into the
review run, and removed the `tend-mention` hand-off. The skill still
treated the review as the last step on a self-authored PR, though. The
**Self-authored PRs** paragraph in Submit said "Submit as `COMMENT` when
there are concerns", and it pointed at **Push fixes** only "after
earlier rounds". The rule to apply the findings sat in step 9, after
Monitor CI and thread resolution. On
[max-sixty/worktrunk#4279](max-sixty/worktrunk#4279),
the reviewer wrote that it would "post this as a comment since it's a
self-authored PR", posted, and ended. It named two options, one of them
a docs-only fix, and deferred both because the other needed a new flag.

The pre-flight now decides who the author is and emits `author: self |
bot | human` in place of `self_authored`. `self` is matched against the
bot login, because the bot's own account is a plain user to GitHub.
`bot` is GitHub's `is_bot` flag (Dependabot, renovate). The skill forks
on that field among the step-1 snapshot fields that decide how much of
the workflow runs. On `self` or `bot`, the run is the author: it posts
the review, then applies the findings in one push.

Three wording changes follow from the fork. The self-authored paragraph
now continues into **Push fixes** instead of ending at the `COMMENT`.
The two "offer to push a fix commit" lines now defer to **Push fixes**,
which already branches on the author. Step 9 adds one sentence: an
option that needs maintainer intent doesn't block one that doesn't.

This failure is uncommon. Of 20 recent worktrunk bot PRs with a
self-review, 16 got later commits, and the three I timed landed 17–36s
after the review, from the review run itself. Two cold A/B scenarios on
the worktrunk#4279 state pushed a fix in every run under both the old
and new text (7/7 each). So this change rests on the structural argument
rather than a measured effect.

> _This was written by Claude Code on behalf of max-sixty_

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants