Skip to content

feat(langchain): run agent routes on createAgent; returnDirect ends only on success - #820

Merged
blove merged 2 commits into
mainfrom
blove/create-agent-migration
Sep 23, 2026
Merged

feat(langchain): run agent routes on createAgent; returnDirect ends only on success#820
blove merged 2 commits into
mainfrom
blove/create-agent-migration

Conversation

@blove

@blove blove commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Summary

Agent routes now run on LangChain's createAgent instead of LangGraph's deprecated createReactAgent, and a returnDirect tool ends the run only when it succeeds. A failed call (the tool threw, or the model's arguments failed its schema) goes back to the model so it can correct the call and retry; the first success ends the run. This unblocks validating tools such as hashbrown's invoicing render tool (liveloveapp/hashbrown#575), which could not adopt returnDirect because an invalid_ui rejection ended the run.

Both prebuilt agents route a returnDirect result to the end by tool name alone (upstream fixes proposed in langchain-ai/langgraphjs#2879 and langchain-ai/langchainjs#11712). createAgent's middleware lets B4 route it itself, so this does not wait on upstream. It reverses the caveat #811 documented.

How

What createReactAgent did through options moves into B4 middleware (packages/langchain/src/agent-middleware.ts):

Before (createReactAgent) After (createAgent)
returnDirect flag on the LangChain tool, routed by name Flag cleared; beforeModel (canJumpTo: ["end"]) jumps to end on a successful returnDirect result (endsOnReturnDirect)
ToolNode turns tool errors into status: "error" results wrapToolCall produces the identical message; without it createAgent treats an escaping tool error as fatal and sets no status
Function prompt re-renders fragments from live state wrapModelCall sets the system message from live state
preModelHook summarization → llmInputMessages beforeModel writes the running summary and condensed view (every turn, so no stale view); wrapModelCall sends it
stateSchema as Annotation.Root StateSchema of ReducedValues

Two middlewares, split by what they may write: a beforeModel node writes back every field its middleware declares, which would re-apply an append reducer, so the node-free model/tool middleware reads route fields and the loop-entry middleware declares only replace-semantics fields. The loop-entry middleware exists only for routes that summarize or have a returnDirect tool, so ordinary routes keep the same supersteps per loop against recursionLimit. The system prompt is passed as a SystemMessage so the provider payload stays a plain string (a string systemPrompt becomes a content block in createAgent).

Found along the way

  • Failed turns still write. LangGraph persists checkpoints asynchronously; the checkpoint owning a parked interrupt can be put after a failed run's stream rejects. /runs/wait's failure arm read the checkpoint at rejection and missed the park (pending-interrupts-endpoint "keeps gating when a parked /runs/wait turn then fails"); on main it passes by ~0.2 ms. New trackCheckpointWrites wraps the graph's checkpointer and the adapter drains in-flight writes before a failure propagates, which makes the gate's "the route is done writing" assumption true on Postgres too.
  • Vercel bundle. createAgent imports initChatModel, whose import(config.package) the self-contained-function validator rejects. The Vercel esbuild plugin rewrites that one expression into a clear failure (B4 always passes a model instance) and fails the build if a future langchain changes it.
  • One @langchain/langgraph copy. A zod ^4.4.3 dependency first resolved to 4.6.5 and split langgraph into two peer-variant copies, so the CLI's Command failed instanceof and resumes stopped resuming. The lockfile keeps zod 4.4.3; @b4run/cli declares langchain/zod dev deps so resolution matches.

Dependencies

@b4run/langchain gains langchain ^1.5.12 and zod ^4.4.3; its @langchain/core peer floor rises from ^1.1.47 to ^1.2.12 (what langchain requires). Patch changeset for @b4run/langchain, @b4run/core, @b4run/cli.

Test plan

  • return-direct-graph.test.ts through the real materialized graph: success ends after one model call; a throwing tool and schema-invalid args each go back to the model and the retry ends the run; without returnDirect the model closes. Mutation-checked in both directions (old name-only routing fails the error cases; removing endsOnReturnDirect fails the success case).
  • agent-middleware-graph.test.ts: fragments render from live state; an append field is applied once with the loop-entry node present; summarization sends the condensed view and keeps the running summary.
  • checkpoint-writes.test.ts: drains in-flight and just-issued writes.
  • pnpm build, lint, typecheck, check-docs, check:build-cache, check:release-inventory, test:release-integrity, pack:check
  • pnpm test: 7601 passed; the only 2 failures are Docker round-trips in vercel-target.test.ts ("streams Agent Protocol requests …"), which time out identically on main in this environment
  • CI validate, harness lanes, vercel-native, copilotkit-examples-e2e

🤖 Generated with Claude Code

@blove @claude
feat(langchain): run agent routes on createAgent; returnDirect ends o…
c7e3d75 
…nly on success

Agent routes now materialize with LangChain's `createAgent` instead of
LangGraph's deprecated `createReactAgent`. Both prebuilt agents route a
`returnDirect` tool's result to the end of the graph by tool name, so a
failed call (the tool threw, or the model's arguments failed its schema)
ended the run and the model never got to retry (upstream:
langchain-ai/langgraphjs#2878, langchain-ai/langchainjs#11711).
`createAgent`'s middleware lets B4 route these itself: the converter's
flag is cleared on the LangChain tool, and a `beforeModel` hook jumps to
the end only when this turn's tool results include a successful result
from a `returnDirect` tool.

What `createReactAgent` did through options moves into B4 middleware:

- prompt fragments re-render from live route state in `wrapModelCall`;
- summarization splits into a `beforeModel` hook (running summary plus
  the condensed view, written every turn) and `wrapModelCall`, which sends
  that view;
- `wrapToolCall` returns tool errors to the model as `status: "error"`
  results in LangGraph's exact `Error: … Please fix your mistakes.` form
  (without it `createAgent` treats an escaping tool error as fatal, and its
  own error messages carry no status);
- route state is a `StateSchema` of `ReducedValue`s (`createAgent` does not
  take an `Annotation.Root`).

The model/tool middleware has no graph node and the loop-entry middleware
exists only for routes that summarize or have a `returnDirect` tool, so
ordinary routes keep the same supersteps per loop against `recursionLimit`.
The system prompt is handed over as a `SystemMessage` so the provider
payload stays a plain string rather than a content block.

Found along the way:

- LangGraph persists checkpoints asynchronously, and the checkpoint that
  owns a parked interrupt can be put after a failed run's stream rejects.
  `/runs/wait`'s failure arm read the checkpoint at rejection and could miss
  the park; it passed on `createReactAgent` by microseconds.
  `trackCheckpointWrites` wraps the graph's checkpointer and the adapter
  drains in-flight writes before a failure propagates.
- `createAgent` imports `initChatModel`, whose `import(config.package)` the
  Vercel function validator rejects. The Vercel esbuild plugin rewrites that
  one expression into a clear failure and fails the build if it changes.
- The lockfile keeps zod 4.4.3 and `@b4run/cli` declares `langchain` and
  `zod` dev dependencies, so one `@langchain/langgraph` copy serves the CLI
  and `@b4run/langchain` (two copies broke `instanceof Command` on resume).

`@b4run/langchain` now depends on `langchain` ^1.5.12, and its
`@langchain/core` peer floor rises to ^1.2.12. Docs: `returnDirect` ends
the run only on success, so a validating tool can use it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Sep 23, 2026 •
edited
Loading

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
b4-run Ready Ready Preview Sep 23, 2026 8:14pm UTC

Request Review

github-actions[bot]
github-actions Bot approved these changes Sep 23, 2026
View reviewed changes

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated approval: this PR received an intelligent (AI) code review. See the review comments on this PR.

@blove @claude
fix(langchain): skip the write drain on cancel; bump the vercel-nativ…
755a651 
…e fixture's LangChain pins

A cancelled turn's settle already waits for the abandoned route to unwind,
and the drain there waited on checkpoint writes chained behind the
still-running tool, holding the turn open until the tool finished (the
"parked /runs/wait turn is cancelled mid-flight" gate test failed 10/15
locally). Drain only on a genuine failure.

The native Vercel fixture pinned @langchain/core 1.2.5, below the
^1.2.12 langchain requires, so its remote `b4 build` failed to import.
Pin core 1.2.12, langgraph 1.4.17 and langgraph-checkpoint 1.1.5, the
versions the workspace uses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@blove
blove merged commit ef4c901 into main Sep 23, 2026
30 of 31 checks passed
@blove
blove deleted the blove/create-agent-migration branch September 23, 2026 20:35
@blove blove mentioned this pull request Sep 23, 2026
Merged
2 tasks done
blove added a commit that referenced this pull request Sep 23, 2026
@blove @claude
fix(web): keep recorded lastmod dates while content is unchanged; rem…
ae61f3f 
…ove the dead main job (#823)

The seo-lastmod workflow was meant to regenerate the manifest on main so
PRs never touch it, but it has never pushed a commit: main's branch
protection requires the validate check, so every push was rejected (24
failures; the 11 successes had nothing to push). PRs kept regenerating
the manifest, and because dates came straight from git history, a squash
merge made main's manifest look stale in any full-history checkout and
the next regenerating PR rewrote unrelated routes' dates.

A route whose content digest matches its recorded entry now keeps that
entry exactly; git is read only for routes that changed or are new, and
removed routes drop out. So a squash merge cannot move a date, PRs only
rewrite the entries for pages they changed, and seo:lastmod:check passes
on main with full history.

The workflow is deleted along with its audited fixture entries, and
AGENTS.md says plainly that content PRs regenerate the manifest after
committing their content. The regenerated manifest updates /docs/cli and
/docs/tools, which #820 edited without regenerating.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@blove blove mentioned this pull request Sep 24, 2026
Merged
6 tasks done
blove added a commit to liveloveapp/hashbrown that referenced this pull request Sep 25, 2026
@blove @claude
feat(invoicing): move to B4 0.12.0 and end the run on a successful re…
e4c8d37 
…nder (#585)

## Summary

Bumps the invoicing server's seven `@b4run/*` pins from 0.11.0 to 0.12.0
and has the `render` tool adopt `returnDirect`.

#575 moved to 0.11.0 but deliberately left `render` without
`returnDirect`: in 0.11.0 the flag ended the run on *any* result from
the tool, including an `invalid_ui` rejection, so the model never got
the retry the tool description promises. B4 0.12.0
(cacheplane/b4run#820) moves agent routes to LangChain's `createAgent`
and routes `returnDirect` itself: **a successful call ends the run, a
failed call goes back to the model**. That is exactly `render`'s
contract, so:

- `src/app/assistant/tools/render.ts` exports `returnDirect = true`. A
valid render ends the run without the closing model turn (~2 to 4 s, per
the 0.11.0 measurement); a rejected render still goes back to the model,
which fixes the named component and renders again.
- `evals/harness.spec.ts`: the scripted-answer test no longer scripts a
closing reply and expects an empty final message (a leftover model turn
would now fail the replay for lack of a fixture). The retry-contract
test from #575 stays, now asserting the fixed render ends the run the
same way.

The route middleware's `after` hook already returned an empty final
message for validated runs, so the browser, which renders from the
`render` call's own arguments, sees no difference beyond the run
finishing sooner.

Lockfile: the stale `@b4run/*` entries were removed before `npm install`
(the ERESOLVE trap from the 0.10.0 bump). One copy each of
`@langchain/core` 1.2.12 (satisfies 0.12.0's new `^1.2.12` peer floor)
and `@langchain/langgraph` 1.4.17 is installed; `langchain` 1.5.12
arrives as a new transitive dependency of `@b4run/langchain`.

Also updates `examples/invoicing/README.md`: the 0.12.0 pin, how a
successful render now ends the run, and the eval section's note on
`run.finalMessage` and pre-change fixtures. Review feedback:
`returnDirect` is documented with a TSDoc block (AGENTS.md).

## Test plan

- [x] `tsc --noEmit -p examples/invoicing/server/tsconfig.json`
- [x] `vitest run --config examples/invoicing/server/vitest.config.mts`:
26 files, 223 tests
- [x] `eslint examples/invoicing/server`: 0 errors
- [x] `nx run invoicing-server:eval` (replay, no network): 9 cases, mean
1.00, gate passed. Fixtures recorded before this change still hold the
model's closing call; replay leaves it unused.
- [x] Live run against `gpt-5-mini` in the React client ("Which
customers have invoices more than 90 days overdue, and how much do they
owe?"): `ledgerSummary`, three `aging` calls, then `render` streamed as
1,007 `TOOL_CALL_ARGS` deltas; `RUN_FINISHED` (success) arrived in the
same millisecond as `render`'s `TOOL_CALL_RESULT`, with no closing model
turn and no text chunks. The panel shows the prose, both over-90 invoice
tables and both aging summaries, and no "could not finish" alert.
- [x] `b4 typegen`: `render`'s tool description still comes from the
default export's doc block (the new `returnDirect` TSDoc does not leak
into it).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@blove blove mentioned this pull request Sep 25, 2026
Closed
blove added a commit that referenced this pull request Sep 28, 2026
@blove @claude
test(cli): approval grants through a real agent route
7d9584e 
The endpoint test seeds grant rows against a stub graph and a canned
pending write, and the core park-site test drives a bare compiled graph,
so nothing showed that the minter the adapter injects into
config.configurable still reaches the permission gate when the tool runs
inside the agent loop, which #820 rebuilt on langchain createAgent.

A real agent() route now parks an approve-gated tool through aimock, the
grant is disclosed on pending_interrupts, one resume runs the tool once,
a replay is refused with stale_interrupt without running it again, and a
resume without the grant is refused with grant_required. Both fail when
the adapter stops injecting the minter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@blove blove mentioned this pull request Sep 28, 2026
Merged
blove added a commit that referenced this pull request Sep 28, 2026
@blove @claude
feat(approvals): single-use grants bound to the parked tool call (#745)
17f16ea 
* feat(approvals): single-use grants bound to the parked tool call

B4.run parked a tool call for human approval and resumed it when an answer
arrived, but said nothing about how many times. The parked interrupt was
addressed by `interruptId` and `resumeKey`, and neither is a credential:
`interruptId` is a millisecond timestamp plus ~31 bits of `Math.random`,
disclosed in the persisted envelope, and `resumeKey` is LangGraph's
deterministic XXH3 of the checkpoint namespace. `ThreadAccessPolicy` gates
who may touch a thread, but disclosure control is not consumption control —
inside a session that legitimately held the thread, nothing stopped an
approval being replayed, or a stale approval from an earlier proposal being
applied to the current one. The only defence was emergent from LangGraph
advancing the checkpoint; nothing enforced or tested it.

A grant is 32 CSPRNG bytes minted at the park site in `@b4run/core`, stored
only as a SHA-256 hash in a B4-owned `interrupt_grants` table added by an
additive versioned migration under the existing `runMigrations` advisory
lock. It rides out on the channels that already carry the prompt and comes
back as an opaque `grant` on the resume entry. Consumption is a conditional
`UPDATE ... WHERE consumed_at IS NULL AND voided_at IS NULL`, so the row
count decides the winner: atomic, durable across a restart, correct across
replicas. A replay is refused rather than re-executed.

The minter is injected through LangGraph's `config.configurable` — the same
channel this repo already uses for live per-call identity — so nothing in
core's call graph grows a storage handle. That injection is optional by
construction, and the absence fails closed: under `approvals.grants:
"required"`, a park with no minter aborts the turn instead of parking a
prompt nobody can answer safely. The mode deliberately does NOT travel the
same way; a mode that went missing alongside the minter could never detect
the minter's absence.

Off by default. Refs #736, #738.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(cli): approval grants through a real agent route

The endpoint test seeds grant rows against a stub graph and a canned
pending write, and the core park-site test drives a bare compiled graph,
so nothing showed that the minter the adapter injects into
config.configurable still reaches the permission gate when the tool runs
inside the agent loop, which #820 rebuilt on langchain createAgent.

A real agent() route now parks an approve-gated tool through aimock, the
grant is disclosed on pending_interrupts, one resume runs the tool once,
a replay is refused with stale_interrupt without running it again, and a
resume without the grant is refused with grant_required. Both fail when
the adapter stops injecting the minter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(web): regenerate SEO lastmod for the approval-grants docs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(changeset): approval grants take patch bumps on the 0.x fixed group

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This was referenced Sep 28, 2026
Closed
Merged

This branch was successfully deployed

1 active and 1 inactive deployments
Preview — 755a6517 Deployed Sep 23, 2026 by vercel[bot]
View deployment
vercel-preview — 755a6517 Deployed Sep 23, 2026 by blove via vercel-native #2315
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actions github-actions[bot] github-actions[bot] approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@blove