Repository navigation
[AISAGE-774] feat(byok): serve Kilo catalog model ids through direct BYOK keys in catalog mode - #8918
chrarnoldus wants to merge 7 commits into
Conversation
Code Review SummaryStatus: No Issues Found | Recommendation: Merge Executive SummaryThe incremental change ( Assumptions: Files Reviewed (3 files)
Previous Review Summaries (6 snapshots, latest commit 1bb7f3a)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit 1bb7f3a)Status: 1 Issue Found | Recommendation: Address before merge Executive SummaryCatalog-mode routing, organization-policy enforcement, key-management authorization, and stored-reference translation look sound; the one new issue is a consistency gap where the availability/offer helpers skip the live-catalog check the routing helpers use, so Assumptions: Overview
Issue Details (click to expand)SUGGESTION
Files Reviewed (28 files)
Fix these issues in Kilo Cloud Previous review (commit e1de0be)Status: No Issues Found | Recommendation: Merge Executive SummaryThis incremental pass re-checked the catalog-mode PR rebased onto Assumptions: Files Reviewed (30 files)
Previous review (commit 4dcb974)Status: No Issues Found | Recommendation: Merge Executive SummaryThis incremental pass re-checked the catalog-mode PR at Assumptions: Files Reviewed (29 files)
Previous review (commit e3f25ab)Status: No Issues Found | Recommendation: Merge Executive SummaryThe incremental refactor ( Assumptions: Files Reviewed (15 files)
Previous review (commit fd9cee4)Status: No Issues Found | Recommendation: Merge Executive SummaryThe three follow-up fixes — basing the key-dialog catalog notice on the edited key's mode, preserving every stored original of a translated favorite, and checking the key's API support before resolving an Organization Auto catalog target — are each correct at current HEAD, and the previously flagged MCP exposure was retained on purpose with the same owner/admin authority as the existing allow-list mutation. Assumptions: Files Reviewed (5 files)
Previous review (commit 73f5426)Status: 4 Issues Found | Recommendation: Address before merge Executive SummaryCatalog mode is wired coherently through routing, organization policy, listings, and key management; the findings are a misleading catalog-mode notice in the shared add/edit key dialog, a new role-gated MCP write exposure for Assumptions: Overview
Issue Details (click to expand)WARNING
SUGGESTION
Verified: catalog routing sets Files Reviewed (33 files)
Reviewed by deepseek-flash-latest · Input: 40.5K · Output: 4.7K · Cached: 351.9K Review guidance: REVIEW.md from base branch |
e517884 to
e1de0be
Compare
…BYOK keys in catalog mode
…odels, store it in key settings
e1de0be to
1bb7f3a
Compare
# Conflicts: # packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.test.ts
Summary
Prep PRs already on
main:byok_api_keys.settingsmigration.This PR builds on them.
Implements the plan in #8497. #8838 already merged the storage step (models.dev
canonical_model_id→catalog_model_idon synced direct BYOK models). This PR adds catalog mode: a direct BYOK key that serves Kilo catalog model IDs such asmoonshotai/kimi-k2.6, with organization model policies enforced, instead of bespoke IDs such assynthetic/hf:moonshotai/kimi-k2.6.Data and storage
Uses the
byok_api_keys.settingscolumn (jsonb not null default '{}', typed byByokApiKeySettingsSchema). The migration shipped separately in [AISAGE-774] feat(db): add byok_api_keys.settings jsonb column #8927, so this PR contains no migration. It holds the flag assettings.serve_catalog_modelsand leaves room for future per-key settings. Reads parse the column, and writes merge into it with||, so other keys are preserved.Catalog mode is off by default, for new and existing keys alike.
Catalog mode is only offered for a direct BYOK provider that has a
models_dev_idand currently maps at least one model to a catalog ID. It is never offered for Vercel BYOK providers, or for providers such asmorph-byokthat map no model. The router refuses to turn it on anywhere else.Primary selection (
direct-byok/catalog-models.ts): when several upstream models of one provider map to one catalog ID, only one is served under it. The order is:recommendedmodel;The map is derived from the cached model list and memoised per list.
A mapping only counts while its catalog ID is in the current OpenRouter snapshot (
isValidOpenRouterModelId). This applies to routing, model-list badges and whether a provider offers catalog mode at all. Kilo-exclusive models are never served from a user's key.Routing and policy (
get-provider.ts,llm-proxy.ts)checkDirectBYOKfinds the primary model of each provider that serves the ID, inDIRECT_BYOK_PROVIDERSorder. It uses the first provider where the caller (user or organization) has an enabled catalog-mode key.bypassAccessCheck: falsewithuserByokset. The organization model policy therefore applies to the catalog ID, and the zero-balance block is skipped, as for Vercel BYOK.direct-byok-catalog-onlyresult, and the proxy fails fast with a 400 and error typedirect_byok_catalog_only. The message names the catalog ID to use, or says the model has no catalog ID.Model list and validation
hasUserByokAvailable: true, in both the personal and the organization list./models/validateand custom-router entry validation reject them as a result, because both read the same list.Key management (
byok-router.ts,BYOKKeysManager.tsx)list,create,updateandsetEnabledreturn three new fields:serve_catalog_models,supports_catalog_mode(the provider currently maps at least one model) andcan_change_catalog_mode.setCatalogModemutation:ORGANIZATION_MANAGE_ROLES), in both directions. Changes are audit-logged.services/kilo-mcp/catalog.jsonpublishesbyok.setCatalogMode. They apply the same role checks as in the app, the same asorganizations.settings.updateAllowLists.Stored references
removeFavoritealso removes the stored originals that were read as that ID.setFavoriteskeeps every stored original in place of its translation and keeps hidden ones, so turning catalog mode off restores them.organizationId. Mobile now sends it onremoveFavorite.validateOrganizationAutoTargetfirst checks that the key's upstream supports the request's API. It then resolves a bespoke target of a catalog-mode key to its catalog ID and applies the model policy to that ID. Saving a route therefore stores the catalog ID, and stored bespoke targets keep resolving at request time. An unmapped target gets a specific error.Changes from the plan
These address the open review remarks on #8497 or keep the change smaller:
settingscolumn rather than a boolean column. Because the default is off, the coding-plan insert path never sets it.Verification
mainafter [AISAGE-774] feat(db): add byok_api_keys.settings jsonb column #8927, [AISAGE-774] refactor(byok): share BYOK key columns and response mapping in the router #8988 and [AISAGE-774] refactor(byok): look up BYOK keys through owner-scoped helpers #8989.packages/dbis identical tomain. Conflicts were resolved against the prep PRs, andcheckDirectBYOKnow usesmain's name,resolveDirectByokProvider.pnpm test:db, then 202 affected Jest suites: 4792 passed. The one failure, inorganizations.test.ts, passes when run alone and is unrelated.web-shared,webandai-gatewaytypecheck,oxlint,oxfmt --checkand the MCP catalog check pass.pnpm script src/scripts/mcp-catalog/dump.ts;-- --checkpasses after the rebase.pnpm --filter @kilocode/web-shared typecheck,pnpm --filter web typecheckandpnpm --filter ai-gateway typecheckpass.scripts/typecheck-all.sh --changes-onlyfails only inapps/mobile, on files this PR does not touch (missing local@kilocode/harness-sdkbuild).oxlint --config .oxlintrc.jsononapps/web/src,packages/web-shared/src,apps/ai-gateway/srcandpackages/db/srcis clean. The mobile config onapps/mobile/src/lib/hooksis also clean.pnpm exec oxfmton changed files.apps/web(DB-backed suites against the local test database), 80 suites, 1904 passed. Suites run: direct BYOK, BYOK, handlers, organizations,get-provider, auto-model,model-allow,byok-router,model-preferences-router,openrouter-models, auto-routing pool validation and routes, direct BYOK sync, organization routers, and balance-check eligibility.checkDirectBYOKfor rejected bespoke IDs with and without a mapping, catalog routing through the first catalog-mode key with access checks on, and fall-through;setCatalogModepermissions, audit log and defaults;inventory-validation: 72 passed. Mobileuse-model-preferences.test.ts: 6 passed.settings: 184 affected suites pass (4366 tests).web-shared,webandai-gatewaytypecheck andoxlintpass.model-preferences-router,organization-auto-model, BYOK UI andmcp-catalogsuites, 125 passed.web-sharedandwebtypecheck andoxlintpass.Models used:
anthropic/claude-opus-5.5(Kilo), no subagent model overrides.