Repository navigation
feat(auto-routing): serve custom routers on the CLI and gateway paths - #7791
Merged
Merged
Conversation
Adds ADR 0005 shared contracts: custom router model IDs, pool entry schemas, the public decide API, client eval protocol schemas, web REST schemas, and the signed auto-decision token. AutoRoutingModeSchema moves next to AutoRouterSchema so the new module does not import the package entry.
Inside the accuracy band a candidate whose estimated request cost is exactly 0 outranks every paid candidate and drops the cost term (ADR 0005), ordered by effectiveLatency^1.5 / wilson^15. Paid ordering is unchanged. Bumps RANKING_POLICY_VERSION so tables republish.
Stores ADR 0005 custom router definitions with an XOR owner check, one unique slug per owner, and entries typed by the contracts pool entry union. Soft delete removes a user's own routers.
Consumers that typecheck packages/db source would otherwise pull in the contracts source, which needs a newer TypeScript lib than some of them set (services/ai-attribution failed on toSorted). The jsonb entry union mirrors the contracts shape, so call sites that pass contracts values still fail to compile if the shape drifts.
…ical An entries update must now send defaultEntryKey so the stored default cannot fall outside the new pool, and gateway model IDs must not start with the reserved client: prefix so a candidate model string maps to exactly one entry.
ADR 0005. The auto-routing worker reads custom router definitions from
PostgreSQL through Hyperdrive with an isolate cache and decides within the
router's pool:
- POST /v1/decide authenticates the caller's Kilo token, resolves the router
by scope or organization context, checks membership and the paying owner's
balance, decides, and returns a signed decision ID.
- POST /v1/eval/{plan,case,result} check router editor rights and the owner's
balance, then forward the router context and grader billing to the
benchmark worker.
- The internal /decide accepts an optional customRouter reference and
returns entry, stale, and fallback; platform requests are unchanged.
- Null decisions use the default entry, then the first servable entry, and
fail with custom_router_unavailable otherwise.
Token pepper, organization context, membership, member balance, the paid
usage balance gate, and the enterprise deny list move to
@kilocode/worker-utils/account-access; the gateway and web callers use it.
![kilo-code-bot[bot]](https://avatars.githubusercontent.com/in/2193792?s=60&v=4){kind=small}
Contributor
Code Review SummaryStatus: No Issues Found | Recommendation: Merge Executive SummaryThe follow-up commit narrows the denied-models retry to transient failures by carrying the gateway status on a dedicated error, retries only network errors, timeouts, 429, and 5xx, and adds a test that a 404 fails closed after a single gateway call; both changed files were reviewed and no new issues were found on the changed lines. Files Reviewed (2 files)
Previous Review Summaries (4 snapshots, latest commit a7d0ca2)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit a7d0ca2)Status: 1 Issue Found | Recommendation: Address before merge Executive SummaryThe follow-up commit adds retryable policy failures, an Upstash command-form read, local dev wiring, and a client-entry fallback; all fail-closed behavior is preserved, and the only finding is a suggestion that the new retry also re-issues requests that can never succeed. Overview
Issue Details (click to expand)SUGGESTION
Verification of prior findings
Notes
Files Reviewed (8 files)
Fix these issues in Kilo Cloud Previous review (commit e688f22)Status: No Issues Found | Recommendation: Merge Executive SummaryThe incremental commit addresses both prior suggestions correctly — the blocked-domain check now fails closed on a cold isolate and the unreachable deny-list branch is removed with behavior preserved — with no new correctness, auth, or fail-closed regressions found. Verification of prior findings
Notes
Files Reviewed (5 files)
Previous review (commit 4c78788)Status: 2 Issues Found | Recommendation: Address before merge Executive SummaryThe incremental changes are behavior-preserving refactors plus two best-effort additions (the organization model policy and blocked-domain checks on the CLI path); the only findings are two low-severity suggestions, with no correctness, auth, or fail-closed regressions. Overview
Issue Details (click to expand)SUGGESTION
Files Reviewed (23 files)
Notes
Fix these issues in Kilo Cloud Previous review (commit 5b20174)Status: 1 Issue Found | Recommendation: Address before merge Executive SummaryThe custom-router serving path and the shared Overview
Fix these issues in Kilo Cloud Issue Details (click to expand)SUGGESTION
Files Reviewed (40 files)
Notes
Reviewed by deepseek-v4.1-flash · Input: 35.9K · Output: 8.8K · Cached: 346.1K Review guidance: REVIEW.md from base branch |
…eat/auto-custom-routers-serve
…domains on the CLI path The public decide route now asks the AI gateway which of a router's gateway entries the organization model policy denies the member. A new internal gateway route runs the gateway's own evaluation (group policy grants, provider limits, and the data-collection rule) for named candidates, so the CLI path and the gateway share one implementation. The worker caches the answer for a minute and denies every gateway entry when the gateway cannot answer. Blocked email domain parsing and matching move to @kilocode/worker-utils/account-access. Every public worker route rejects a user in a blocked domain, read from the same Upstash key the web app uses.
…drop an unreachable deny-list branch
This was referenced Oct 16, 2026
…ryable policy failures - .dev.vars.example points KILO_AI_GATEWAY_BASE_URL at the local AI gateway and seeds the Upstash secrets from the dev redis-http container; .env.local.example and ENVIRONMENT.md document AUTO_ROUTING_DECISION_SECRET and the worker's Upstash secrets. - The blocked-domain read uses the POST ["GET", key] command form that Upstash and serverless-redis-http both serve. - A failed denied-models lookup is retried once. If it still fails, listed client entries can serve; otherwise the request gets a retryable internal_error with Retry-After instead of custom_router_unavailable.
pandemicsyn
approved these changes
Oct 19, 2026
…ve-ready # Conflicts: # docs/adr/0005-custom-auto-routers.md # packages/auto-routing-contracts/src/custom-router.ts
This was referenced Oct 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the serving side of ADR 0005. Stacked on #7784.
What changes
Auto-routing worker
auto_routing_custom_routersthrough the existing Hyperdrive binding, with a short isolate cache.POST /v1/decide(public, Kilo token):X-KiloCode-OrganizationIdor the token's signed organization). The worker checks membership.signAutoDecision).POST /v1/eval/{plan,case,result}(public):BENCHMARK_SERVICE. The benchmark worker's status and body pass through./decideaccepts the optionalcustomRouterandclientEntriesand returnsentry,stale, andfallback. Requests withoutcustomRoutertake the same path as before.custom:<routerId>.custom_router_unavailable. The worker never routes outside the pool.requested_model.Shared access rules: new module
@kilocode/worker-utils/account-access, used by both the worker and the gateway. It holds:The gateway auth, balance, and blocked-domain code and the web callers of
isOrganizationMembernow use it. Gateway routing behavior is unchanged.Organization model policy on the CLI path: the full policy (group-policy grants, provider limits, data-collection rule, direct-BYOK exemptions) depends on the gateway's provider runtime: provider definitions, model snapshots, and direct-BYOK model lists. Instead of copying it, a new internal gateway route
POST /api/v1/internal/auto-routing/denied-models(internal secret) runs the gateway's own evaluation for named candidates. The CLI decide path sends it the router's gateway entries in an organization context, caches the answer for a minute, and denies every gateway entry when the gateway cannot answer. Every public worker route also rejects users in a blocked email domain.Deploy notes
NEXTAUTH_SECRET_PROD(already exists; now bound to this worker),AUTO_ROUTING_DECISION_SECRET_PROD(new; the gateway gets the same value in the next stacked PR), andAUTO_ROUTING_UPSTASH_REDIS_REST_URL_PROD/AUTO_ROUTING_UPSTASH_REDIS_REST_READONLY_TOKEN_PROD(new; the web app's Upstash database, with a read-only token).KILO_AI_GATEWAY_BASE_URLpoints at the production gateway. Deploy the gateway route before the worker, because the worker fails closed on gateway entries in organization contexts until the route exists.Known limits
BLACKLIST_DOMAINSenv fallback, which applies when Redis holds no list, is a Vercel env var that the worker does not have.Verification
services/auto-routing: typecheck, full vitest suite.packages/worker-utils: typecheck, full vitest suite.packages/auto-routing-contracts: vitest.packages/web-sharedandapps/web: typecheck.isOrganizationMembercallers, including the DB-backed organization usage tests.