Skip to content

feat(auto-routing): custom router contracts and schema - #7784

Merged
iscekic merged 11 commits into
mainfrom
feat/auto-custom-routers
Oct 19, 2026
Merged

iscekic merged 11 commits into
mainfrom
feat/auto-custom-routers

Conversation

@iscekic

@iscekic iscekic commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator

Implements PR 0 of ADR 0005 (docs/adr/0005-custom-auto-routers.md): the ADR plus the shared contracts and PostgreSQL schema every later layer builds on. No worker or web behavior here.

Contracts (@kilocode/auto-routing-contracts)

  • custom-router.ts: custom router model IDs with slug validation and the reserved personal org slug, pool entry schemas for gateway and client entries, entry keys, entry refs, and the candidate-model helpers that keep client entries (client:<provider>/<model>) from colliding with gateway IDs in deny lists.
  • Router definitions and the public worker API: POST /v1/decide request/response, the error code enum with its HTTP status table, and the client eval protocol schemas (plan, case, result upload). Golden answers and checks are deliberately absent from every schema.
  • Web REST schemas used by the web UI and the CLI: list, create, partially-applied update, and client-entry registration, with per-entry-variant status.
  • MirrorPayloadSchema gains an optional custom router reference and client entries; AutoRoutingDecisionSchema gains optional entry, stale, and fallback fields so old readers and writers keep parsing during a rolling deploy.
  • Signed decision token: signAutoDecision / verifyAutoDecision over WebCrypto HMAC-SHA256 (Cloudflare Workers and Node), constant-time verification, canonical base64url, and expired/malformed rejection, covered by tamper, expiry, and round-trip tests.

Ranking

Within the accuracy band a candidate whose estimated request cost is exactly zero now outranks every paid candidate and takes no cost term, ordered by effective latency and Wilson accuracy alone. Paid ordering is unchanged. RANKING_POLICY_VERSION is bumped so tables republish, and the benchmark service README documents the rule.

PostgreSQL

New auto_routing_custom_routers table with an exclusive owner check, one unique slug per owner via partial unique indexes, owner lookup indexes, and entries jsonb typed with the contracts entry union. Generated with pnpm drizzle generate; soft delete removes a user's own routers.

Later PRs (benchmark worker, serving worker, web/gateway, cutover) and the CLI build on these contracts.

Adds ADR 0005 shared contracts: custom router model IDs, pool entry
schemas, the public decide API, client eval protocol schemas, web REST
schemas, and the signed auto-decision token. AutoRoutingModeSchema moves
next to AutoRouterSchema so the new module does not import the package
entry.
Inside the accuracy band a candidate whose estimated request cost is
exactly 0 outranks every paid candidate and drops the cost term
(ADR 0005), ordered by effectiveLatency^1.5 / wilson^15. Paid ordering
is unchanged. Bumps RANKING_POLICY_VERSION so tables republish.
Stores ADR 0005 custom router definitions with an XOR owner check,
one unique slug per owner, and entries typed by the contracts pool
entry union. Soft delete removes a user's own routers.
@iscekic iscekic self-assigned this Oct 16, 2026
Consumers that typecheck packages/db source would otherwise pull in the
contracts source, which needs a newer TypeScript lib than some of them
set (services/ai-attribution failed on toSorted). The jsonb entry union
mirrors the contracts shape, so call sites that pass contracts values
still fail to compile if the shape drifts.
@kilo-code-bot

kilo-code-bot Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

No Issues Found | Recommendation: Merge

Executive Summary

The incremental commit 61748246 updates only the ADR 0005 markdown; the new prose (quota admission order, client-eval prompt provenance, organization-model-policy lookup, gateway decision verification, and worker error mapping) is internally consistent with the contracts already shipped in the earlier commits of this PR.

Files Reviewed (1 file)
  • docs/adr/0005-custom-auto-routers.md - documentation-only ADR updates (lines 91-95, 110-120, 152-181, 190-193); no code change and no new issues

Cross-checks performed against current HEAD 61748246:

  • The quota_exceeded (HTTP 429) plus retryAt admission rule matches CustomRouterErrorCodeSchema and CUSTOM_ROUTER_ERROR_HTTP_STATUS in packages/auto-routing-contracts/src/custom-router.ts:220,233,244.
  • The gateway verification claim that the user, organization, model, and variant must match is backed by AutoDecisionPayloadSchema (userId, organizationId, model, variant) at packages/auto-routing-contracts/src/custom-router.ts:299.
  • The error codes the ADR references (forbidden, router_not_found, custom_router_unavailable) exist in the contract enum; auto_routing_unavailable is documented as a gateway-level mapping, not a contract enum member.
  • No images were added or changed, so the markdown-image convention is unaffected; no memory-leak vectors are introduced by a documentation-only change.

Both prior Code Review Findings remain resolved at current HEAD: GatewayCustomRouterEntrySchema reserves the client: prefix (candidate-model injectivity), and UpdateCustomRouterRequestSchema requires entries and defaultEntryKey together.

Previous Review Summaries (5 snapshots, latest commit 984555c)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 984555c)

Status: No Issues Found | Recommendation: Merge

Executive Summary

The incremental commit 984555c4 collapses the unused ClientEvalCaseResponse fields systemPrompt/userPrompt into a single prompt: z.string().min(1); the schema has no consumers in the repo, so the rename is safe and self-consistent, and both previously reported suggestions are confirmed fixed at current HEAD.

Files Reviewed (1 file)
  • packages/auto-routing-contracts/src/custom-router.ts - contract simplification of ClientEvalCaseResponseSchema (lines 442-444); no new issues

Both prior Code Review Findings are resolved in current code:

  • customRouterCandidateModel injectivity: GatewayCustomRouterEntrySchema (line 68-71) now rejects client:-prefixed model IDs, and parseCustomRouterCandidateModel applies the same rule.
  • Partial update invariant: UpdateCustomRouterRequestSchema (line 564-570) rejects an entries-only update, keeping the stored defaultEntryKey inside entries.

The changed prompt field is a public worker→CLI wire field introduced in this same PR with no in-repository reader or writer yet, so the field replacement introduces no breaking change.

Previous review (commit eeb767d)

Status: No Issues Found | Recommendation: Merge

Executive Summary

The incremental commit eeb767d6d adds the quota_exceeded custom-router error code (HTTP 429) and an optional retryAt field to the error response, with a regression test; the change is additive, type-consistent, and introduces no new issues.

Files Reviewed (2 files)
  • packages/auto-routing-contracts/src/custom-router.ts - new quota_exceeded enum member with its 429 status mapping and optional retryAt; the satisfies Record<CustomRouterErrorCode, number> map stays exhaustive and no consumer switches exhaustively on the code
  • packages/auto-routing-contracts/src/custom-router.test.ts - asserts the 429 mapping and parses a quota_exceeded response carrying retryAt

Previous review (commit c18af20)

Status: No Issues Found | Recommendation: Merge

Executive Summary

The incremental commit c18af2091 adds the internal benchmark-worker wire schemas (request context, plan/case/result, status, billing) and gates the newly nullable classification and table fields on AutoRoutingDecisionSchema to fallback decisions only; both are covered by regression tests and introduce no new issues.

Files Reviewed (4 files)
  • packages/auto-routing-contracts/src/custom-router.ts - new ClientEvalPairSchema, CustomRouterBenchmarkContextSchema, internal plan/case/result/status schemas, and the clientEvalEntryKeys refinement (names a pool entry, no duplicates, every client entry listed) match the ADR invariants
  • packages/auto-routing-contracts/src/index.ts - taskType / subtaskType / tableVersion are nullable only when fallback === true; no consumer in the repo relies on their previous non-null type
  • packages/auto-routing-contracts/src/custom-router.test.ts - internal-context tests verified against the refinement logic
  • packages/auto-routing-contracts/src/contracts.test.ts - fallback-nullability test verified against the refine

Previous review (commit f5e4253)

Status: No Issues Found | Recommendation: Merge

Executive Summary

The incremental commit f5e425374 resolves both prior findings — GatewayCustomRouterEntrySchema now rejects models using the reserved client: prefix so candidate-model strings stay injective, and UpdateCustomRouterRequestSchema requires entries and defaultEntryKey together — each with regression tests; the remaining changed lines introduce no new issues.

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 0
Files Reviewed (2 files, incremental)
  • packages/auto-routing-contracts/src/custom-router.ts — both previous findings verified fixed; no new issues
  • packages/auto-routing-contracts/src/custom-router.test.ts — new injectivity and update-pool tests reviewed; consistent with schema behavior

Re-verified since the previous review: the gateway/candidate namespace is now disjoint (client: prefix reserved), parseCustomRouterCandidateModel stays consistent with the gateway branch, and both partial-update directions were checked against the ADR invariant. No memory-leak vectors were introduced by the changed code.

Previous review (commit 2c6a989)

Status: 2 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 2
Issue Details (click to expand)

SUGGESTION

File Line Issue
packages/auto-routing-contracts/src/custom-router.ts 134 Candidate-model namespace is not injective for gateway entries; a gateway model named client:<provider>/<model> collides with a client entry and can never be parsed back.
packages/auto-routing-contracts/src/custom-router.ts 545 UpdateCustomRouterRequestSchema validates defaultEntryKey ∈ entries only when both fields are present, so a partial update can persist a default_entry_key outside the pool.
Files Reviewed (15 files)
  • apps/web/src/lib/user/index.test.ts
  • apps/web/src/lib/user/index.ts
  • docs/adr/0005-custom-auto-routers.md
  • packages/auto-routing-contracts/src/custom-router.test.ts
  • packages/auto-routing-contracts/src/custom-router.ts - 2 issues
  • packages/auto-routing-contracts/src/index.ts
  • packages/auto-routing-contracts/src/router.ts
  • packages/auto-routing-contracts/src/routing-table.test.ts
  • packages/auto-routing-contracts/src/routing-table.ts
  • packages/db/src/migrations/0283_fantastic_annihilus.sql
  • packages/db/src/migrations/meta/0283_snapshot.json
  • packages/db/src/migrations/meta/_journal.json
  • packages/db/src/schema.test.ts
  • packages/db/src/schema.ts
  • services/auto-routing-benchmark/README.md

Also verified: the signed-decision token (canonical base64url, constant-time WebCrypto verify, expiry after signature check), the $0-before-paid ranking comparator (consistent total order, paid and below-band behavior preserved, rankFreeValue matches effectiveLatency^1.5 / wilson^15), and the new auto_routing_custom_routers table (DDL matches the Drizzle schema, empty-table index creation, user soft-delete coverage). No memory-leak vectors were introduced.

Fix these issues in Kilo Cloud


Reviewed by deepseek-v4.1-flash · Input: 38.7K · Output: 4K · Cached: 313.9K

Review guidance: REVIEW.md from base branch main

…ical

An entries update must now send defaultEntryKey so the stored default
cannot fall outside the new pool, and gateway model IDs must not start
with the reserved client: prefix so a candidate model string maps to
exactly one entry.
@iscekic
iscekic added this pull request to stack #7815 October 17, 2026 00:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants