Skip to content

feat(cloud-agent-next): share one scoped credential per worktree - #7491

Merged
eshurakov merged 1 commit into
eshurakov/spry-rapids-2-wrapperfrom
eshurakov/spry-rapids-3-worker
Oct 15, 2026
Merged

eshurakov merged 1 commit into
eshurakov/spry-rapids-2-wrapperfrom
eshurakov/spry-rapids-3-worker

Conversation

@eshurakov

@eshurakov eshurakov commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Merge this PR together with #7490. Neither PR is safe to deploy without the other. Drain old multi-handle allocations and old mixed modern/legacy directory allocations before #7490 merges into main (rollout step 3 in docs/control-plane.md). The scheduled production deploy picks up cloud-agent-next changes on main, so the merge is the deploy.

Summary

Part 3 of 3 in a stack. Base: #7490.

  1. feat(session-ingest): accept worktree-scoped Cloud Agent requests #7489: session-ingest accepts worktree-scoped requests.
  2. feat(cloud-agent-next): key wrapper Kilo runtimes by directory #7490: wrapper, one Kilo runtime per directory.
  3. This PR: Worker, one scoped credential per worktree.

After #7490, the sessions in one worktree share one Kilo runtime. This PR gives them one canonical proxy handle instead of one handle per session.

  • Registration stops writing runtimeIsolation, and the route schema no longer requires it for permissions or profiles. The Sandbox DO removes the field from every outbound session.prepare frame.
  • The Sandbox DO records the accepted handle for each worktree scope. A sibling adopts it, and a rotation replaces it for every member. A handle whose issuer was deleted resolves through a member that is still live. A joiner into a scope that has a handle this fence never accepted fails with stage scope_handle_conflict. Admission rejects mixed credential classes in one directory.
  • The Session DO keeps a wipe locator, so the handle of a deleted issuer can still resolve. Each runtime proxy scope match checks the sandbox id.
  • For an adopted handle, the facade sends the session-ingest v2 scope headers from feat(session-ingest): accept worktree-scoped Cloud Agent requests #7489. The Vercel network policy lets an adopted handle reach every member root.
  • Per-session handles from before this change keep the old path.
  • docs/control-plane.md describes the whole stack, including the rollout and the rollback.

Rollback is not symmetric. If the Worker rolls back while the new wrapper image runs, the old per-session handles share one process. Recycle sandboxes before a rollback.

Verification

  • No manual tests. I did not run the stack on a local or deployed sandbox.

Visual Changes

N/A

Reviewer Notes

  • Size: +7,945/−484. Tests are +4,912 of that. test/integration/control-plane-scope-handle.test.ts alone is 3,108 lines.
  • Local checks on this layer: cloud-agent-next typecheck (Worker and wrapper), unit tests (6,373 pass), wrapper tests (1,185 pass), Worker integration tests (785 pass, 1 fail), check:duplication, oxlint and oxfmt.
  • The one integration failure is sandbox-control-v2-repo-snapshots.test.ts › "cleans up a failed snapshot and launches the image on the same allocation and deadline". It also fails locally on origin/main (8d424cc) without this stack.

@kilo-code-bot

kilo-code-bot Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Full re-review of the worktree-scoped shared-credential stack (41 files) found no high-confidence defects on lines added by this PR; the previous KV-materialization comment in transactional-wipe.ts remains addressed.

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 0

Notes

  • Re-verified the resolved KV-OOM thread in transactional-wipe.ts:61-66: the wipe now iterates kv.list() once and retains only key strings before the delete pass, so full worktree-change values are not snapshotted into memory. No duplicate was filed.
  • Security-critical paths verified: facade caller scope/credential header stripping (stripCallerScopeHeaders removes Authorization, X-Internal-Secret, and every x-kilo-* before the v1/v2 forward sets its own), scope-handle digest/fence membership and constant-time comparison, accepted-handle issuer locator, per-member backing renewal (runtimeProxyScopeMatches checks sandbox/directory/root session), runtime-contract negotiation (missing/stale → legacy, worktree pinned on first authenticated wrapper, downgrade rejected), and Vercel network-policy adoption rules. No auth bypass or cross-scope handle leak identified.
  • No added lines introduce leak-prone patterns: the only new in-memory map (CanonicalHandleMinter.inFlight) deletes its key in finally and is bounded by concurrent scope/fence keys; per-route runtime_proxy_last_sent_handle:* KV entries are deleted in deleteRoute; the accepted-handle store is capped at 4 live entries per scope and retired on wrapper restart, allocation change, and last-member leave.
  • No HTML <img> tags were added to markdown docs, and no TODO/as any/console.log were introduced on added lines.
  • Sub-threshold observations not filed as findings: a stale allocation_runtime_contract record resolves to legacy (bypassing newer guards) rather than re-negotiating, unreachable today because writeAllocation resets the contract on allocation change; and the Vercel policy intentionally omits a descendant export GET rule while the v2 facade resolver accepts it (the scope assertion still bounds it to the worktree).
Files Reviewed (41 files)
  • services/cloud-agent-next/docs/control-plane.md
  • services/cloud-agent-next/scripts/duplication-allowlist.json
  • services/cloud-agent-next/src/cloud-agent-session-scope-proxy.ts
  • services/cloud-agent-next/src/control-plane/sandbox/accepted-handles.ts
  • services/cloud-agent-next/src/control-plane/sandbox/routes.ts
  • services/cloud-agent-next/src/control-plane/sandbox/runtime-contract.test.ts
  • services/cloud-agent-next/src/control-plane/sandbox/runtime-contract.ts
  • services/cloud-agent-next/src/control-plane/sandbox/sandbox-do.ts
  • services/cloud-agent-next/src/control-plane/sandbox/scope-grants.ts
  • services/cloud-agent-next/src/control-plane/sandbox/scope-handle-adoption.ts
  • services/cloud-agent-next/src/control-plane/sandbox/scope-handle-delivery.ts
  • services/cloud-agent-next/src/control-plane/sandbox/scope-handle-resolution.ts
  • services/cloud-agent-next/src/control-plane/sandbox/scope-handle-rotation.test.ts
  • services/cloud-agent-next/src/control-plane/sandbox/scope-handle-rotation.ts
  • services/cloud-agent-next/src/control-plane/sandbox/scope-view-store.ts
  • services/cloud-agent-next/src/control-plane/sandbox/scope-view.ts
  • services/cloud-agent-next/src/control-plane/session/registration.test.ts
  • services/cloud-agent-next/src/control-plane/session/registration.ts
  • services/cloud-agent-next/src/control-plane/session/session-do.ts
  • services/cloud-agent-next/src/control-plane/session/transactional-wipe.test.ts
  • services/cloud-agent-next/src/control-plane/session/transactional-wipe.ts
  • services/cloud-agent-next/src/kilo/runtime-credential-proxy-routes.ts
  • services/cloud-agent-next/src/runtime-credential-proxy.ts
  • services/cloud-agent-next/src/sandbox-control/session-credentials.ts
  • services/cloud-agent-next/src/sandbox-control/vercel-network-policy.ts
  • services/cloud-agent-next/src/sandbox-control/vercel-provider.ts
  • services/cloud-agent-next/src/server.ts
  • services/cloud-agent-next/src/shared/control-plane-protocol.ts
  • services/cloud-agent-next/src/shared/control-plane-runtime-contract.ts
  • services/cloud-agent-next/src/shared/control-plane-timers.ts
  • services/cloud-agent-next/test/integration/control-plane-scope-handle.test.ts
  • services/cloud-agent-next/test/integration/control-plane-session-v2-rpc.test.ts
  • services/cloud-agent-next/test/integration/helpers/fake-sandbox-peer.ts
  • services/cloud-agent-next/test/integration/helpers/fake-session-peer.ts
  • services/cloud-agent-next/test/integration/helpers/fake-wrapper.ts
  • services/cloud-agent-next/test/integration/sandbox-control-v2-credentials.test.ts
  • services/cloud-agent-next/test/integration/sandbox-control-v2-routes.test.ts
  • services/cloud-agent-next/test/integration/sandbox-control-v2-runtime-proxy.test.ts
  • services/cloud-agent-next/test/integration/sandbox-control-v2.test.ts
  • services/cloud-agent-next/wrapper/src/control-plane/connection.test.ts
  • services/cloud-agent-next/wrapper/src/control-plane/connection.ts
Previous Review Summaries (3 snapshots, latest commit 33f76b7)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 33f76b7)

Status: No Issues Found | Recommendation: Merge

Executive Summary

Full review (history was rewritten, so the incremental base was unavailable) of the worktree-scoped shared-credential stack across 41 files found no high-confidence defects on lines added by this PR.

Notes

  • Re-verified the previously raised KV-OOM concern in transactional-wipe.ts: the wipe now iterates kv.list() once and retains only key strings before the delete pass, so full worktree-change values are no longer snapshotted into memory. The existing inline thread is addressed; no duplicate was filed.
  • Verified the security-critical paths: facade caller scope/credential header stripping, runtimeIsolation removal being intentional for negotiated worktree mode, accepted-handle digest/fence membership checks, scope-handle ownership and expiry arithmetic, and fail-closed route resolution. No auth bypass or cross-scope handle leak identified.
  • Verified no added lines contain leak-prone patterns (event listeners, intervals, unbounded caches) or leftover TODO/as any/console.log.
  • Two sub-threshold observations were not filed: a malformed allocation_runtime_contract row throws in the hello path without an explicit close (storage-corruption only), and hasSharedScope() keeps routing to the Sandbox DO past locator expiry until cold init (the Sandbox DO still re-validates and denies).
Files Reviewed (41 files)
  • services/cloud-agent-next/docs/control-plane.md
  • services/cloud-agent-next/scripts/duplication-allowlist.json
  • services/cloud-agent-next/src/cloud-agent-session-scope-proxy.ts
  • services/cloud-agent-next/src/control-plane/sandbox/accepted-handles.ts
  • services/cloud-agent-next/src/control-plane/sandbox/routes.ts
  • services/cloud-agent-next/src/control-plane/sandbox/runtime-contract.test.ts
  • services/cloud-agent-next/src/control-plane/sandbox/runtime-contract.ts
  • services/cloud-agent-next/src/control-plane/sandbox/sandbox-do.ts
  • services/cloud-agent-next/src/control-plane/sandbox/scope-grants.ts
  • services/cloud-agent-next/src/control-plane/sandbox/scope-handle-adoption.ts
  • services/cloud-agent-next/src/control-plane/sandbox/scope-handle-delivery.ts
  • services/cloud-agent-next/src/control-plane/sandbox/scope-handle-resolution.ts
  • services/cloud-agent-next/src/control-plane/sandbox/scope-handle-rotation.test.ts
  • services/cloud-agent-next/src/control-plane/sandbox/scope-handle-rotation.ts
  • services/cloud-agent-next/src/control-plane/sandbox/scope-view-store.ts
  • services/cloud-agent-next/src/control-plane/sandbox/scope-view.ts
  • services/cloud-agent-next/src/control-plane/session/registration.test.ts
  • services/cloud-agent-next/src/control-plane/session/registration.ts
  • services/cloud-agent-next/src/control-plane/session/session-do.ts
  • services/cloud-agent-next/src/control-plane/session/transactional-wipe.test.ts
  • services/cloud-agent-next/src/control-plane/session/transactional-wipe.ts
  • services/cloud-agent-next/src/kilo/runtime-credential-proxy-routes.ts
  • services/cloud-agent-next/src/runtime-credential-proxy.ts
  • services/cloud-agent-next/src/sandbox-control/session-credentials.ts
  • services/cloud-agent-next/src/sandbox-control/vercel-network-policy.ts
  • services/cloud-agent-next/src/sandbox-control/vercel-provider.ts
  • services/cloud-agent-next/src/server.ts
  • services/cloud-agent-next/src/shared/control-plane-protocol.ts
  • services/cloud-agent-next/src/shared/control-plane-runtime-contract.ts
  • services/cloud-agent-next/src/shared/control-plane-timers.ts
  • services/cloud-agent-next/test/integration/control-plane-scope-handle.test.ts
  • services/cloud-agent-next/test/integration/control-plane-session-v2-rpc.test.ts
  • services/cloud-agent-next/test/integration/helpers/fake-sandbox-peer.ts
  • services/cloud-agent-next/test/integration/helpers/fake-session-peer.ts
  • services/cloud-agent-next/test/integration/helpers/fake-wrapper.ts
  • services/cloud-agent-next/test/integration/sandbox-control-v2-credentials.test.ts
  • services/cloud-agent-next/test/integration/sandbox-control-v2-routes.test.ts
  • services/cloud-agent-next/test/integration/sandbox-control-v2-runtime-proxy.test.ts
  • services/cloud-agent-next/test/integration/sandbox-control-v2.test.ts
  • services/cloud-agent-next/wrapper/src/control-plane/connection.test.ts
  • services/cloud-agent-next/wrapper/src/control-plane/connection.ts

Previous review (commit d2a8fe1)

Status: No Issues Found | Recommendation: Merge

Executive Summary

Incremental review of the single fix commit d2a8fe195: the transactional wipe now collects only key strings before deleting, so the previous KV-value OOM concern is resolved. No new defects found.

Files Reviewed (1 file)
  • services/cloud-agent-next/src/control-plane/session/transactional-wipe.ts - 0 issues

Notes: Re-verified the prior WARNING at transactional-wipe.ts:61 against current HEAD. runTransactionalWipe now iterates kv.list() once, retaining only the key strings in a local array before the delete pass, so full worktree-change values are no longer snapshotted into memory during the wipe. The two-pass key-then-delete shape is correct for avoiding mutation during iteration, and the locator/generation skip logic is unchanged. No other changed lines in this increment.

Previous review (commit 0b281fb)

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
services/cloud-agent-next/src/control-plane/session/transactional-wipe.ts 61 Array.from(input.storage.kv.list()) materializes every KV value (including full worktree-change file content) into memory during the wipe, risking an OOM in the 128 MiB DO isolate.
Files Reviewed (34 files)
  • services/cloud-agent-next/docs/control-plane.md - 1 (doc accuracy note, not filed)
  • services/cloud-agent-next/scripts/duplication-allowlist.json - 0
  • services/cloud-agent-next/src/cloud-agent-session-scope-proxy.ts - 0
  • services/cloud-agent-next/src/control-plane/sandbox/accepted-handles.ts - 0
  • services/cloud-agent-next/src/control-plane/sandbox/routes.ts - 0
  • services/cloud-agent-next/src/control-plane/sandbox/sandbox-do.ts - 0
  • services/cloud-agent-next/src/control-plane/sandbox/scope-grants.ts - 0
  • services/cloud-agent-next/src/control-plane/sandbox/scope-handle-adoption.ts - 0
  • services/cloud-agent-next/src/control-plane/sandbox/scope-handle-delivery.ts - 0
  • services/cloud-agent-next/src/control-plane/sandbox/scope-handle-resolution.ts - 0
  • services/cloud-agent-next/src/control-plane/sandbox/scope-handle-rotation.test.ts - 0
  • services/cloud-agent-next/src/control-plane/sandbox/scope-handle-rotation.ts - 0
  • services/cloud-agent-next/src/control-plane/sandbox/scope-view.ts - 0
  • services/cloud-agent-next/src/control-plane/session/registration.test.ts - 0
  • services/cloud-agent-next/src/control-plane/session/registration.ts - 0
  • services/cloud-agent-next/src/control-plane/session/session-do.ts - 0
  • services/cloud-agent-next/src/control-plane/session/transactional-wipe.test.ts - 0
  • services/cloud-agent-next/src/control-plane/session/transactional-wipe.ts - 1
  • services/cloud-agent-next/src/kilo/runtime-credential-proxy-routes.ts - 0
  • services/cloud-agent-next/src/runtime-credential-proxy.ts - 0
  • services/cloud-agent-next/src/sandbox-control/session-credentials.ts - 0
  • services/cloud-agent-next/src/sandbox-control/vercel-network-policy.ts - 0
  • services/cloud-agent-next/src/sandbox-control/vercel-provider.ts - 0
  • services/cloud-agent-next/src/server.ts - 0
  • services/cloud-agent-next/src/shared/control-plane-protocol.ts - 0
  • services/cloud-agent-next/src/shared/control-plane-timers.ts - 0
  • services/cloud-agent-next/test/integration/control-plane-scope-handle.test.ts - 0
  • services/cloud-agent-next/test/integration/control-plane-session-v2-rpc.test.ts - 0
  • services/cloud-agent-next/test/integration/helpers/fake-sandbox-peer.ts - 0
  • services/cloud-agent-next/test/integration/helpers/fake-session-peer.ts - 0
  • services/cloud-agent-next/test/integration/sandbox-control-v2-credentials.test.ts - 0
  • services/cloud-agent-next/test/integration/sandbox-control-v2-routes.test.ts - 0
  • services/cloud-agent-next/test/integration/sandbox-control-v2-runtime-proxy.test.ts - 0
  • services/cloud-agent-next/test/integration/sandbox-control-v2.test.ts - 0

Notes: Reviewed the net diff against 7b3e03c65 (base #7490) at head 0b281fbf6. Verified the security-critical paths: runtime-proxy scope match per sandbox id/directory/kiloSessionId, fence filtering on accepted-handle lookup and scope views, caller scope-header stripping on the facade, the v2 scoped-ingest route allowlist, and the fail-closed route resolution. No other high-confidence defects found; the shared-handle revocation nuance in docs/control-plane.md (scope-wide handle vs. per-member revocation) is consistent with the documented design and was not filed.

Fix these issues in Kilo Cloud


Reviewed by deepseek-v4.1-flash · Input: 187.3K · Output: 25K · Cached: 3.7M

Review guidance: REVIEW.md from base branch eshurakov/spry-rapids-2-wrapper

@eshurakov
eshurakov force-pushed the eshurakov/spry-rapids-3-worker branch 2 times, most recently from a290248 to 33f76b7 Compare October 15, 2026 10:10
Negotiate and persist runtime compatibility per allocation so existing sandboxes keep their sessions and credentials. Enable shared worktree handles for new capable allocations, preserve authorization fences, and document the rolling deployment order.
@eshurakov
eshurakov force-pushed the eshurakov/spry-rapids-3-worker branch from 33f76b7 to 0a2976b Compare October 15, 2026 11:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants