[chrarnoldus]

Summary

Adds a Provider Order tab to /admin/gateway for editing the preferred inference provider order as JSON, using the same Monaco editor as Auto Free. Like the Auto Free, Preferred Models and Vercel model ID override PRs, the config is stored but not used yet: applyPreferredProvider and the catalog pricing in providers/openrouter/index.ts still use the built-in rules.

• DB migration. Migration 0293_preferred_provider_order adds a nullable preferred_provider_order jsonb column to the ai_gateway_config singleton. It is one additive ADD COLUMN with no backfill. null means nothing is stored. It started as 0292 in its own commit. Main later added 0292_direct_byok_sync_state, so the merge commit regenerates it as 0293 with drizzle-kit generate. The DDL is unchanged.
• Schema. PreferredProviderOrderConfigSchema in @kilocode/db/schema-types:

  { "rules": [{ "includes": ["openai", "gpt"], "excludes": ["gpt-oss"], "providers": ["openai"] }] }

  • A rule matches when the requested model ID contains at least one includes substring and none of the optional excludes substrings. Rules are checked in order and the first match sets the order. If nothing matches, there is no preference. excludes covers cases like gpt but not gpt-oss within a single rule.
  • It is an array of rules, not an object keyed by substring, because jsonb does not keep object key order, so a first-match object could not be stored reliably.
  • Limits: substrings are lowercase and use the model ID character set. Provider slugs are lowercase and can have one / for an endpoint variant such as minimax/fp8. Each rule needs at least one include and at least one provider. There are at most 200 rules and 20 providers per rule. Duplicates within includes, excludes or providers are rejected, and so is a substring that is both included and excluded.
• Built-in rules as data. The hardcoded if chain in getPreferredProviderOrder is now PREFERRED_PROVIDER_ORDER_FALLBACK_CONFIG plus resolvePreferredProviderOrder(config, model), in the new providers/preferred-provider-order.ts. The OpenAI rule (openai or gpt, excluding gpt-oss) reproduces isOpenAiModel exactly, so behavior does not change: the existing applyPreferredProvider tests pass without edits. The editor's defaults are exactly what the gateway runs today, and connecting the stored config later only means passing a different config.
• Router. admin.preferredProviderOrder.get returns { config, fallback }. admin.preferredProviderOrder.set upserts only the new column, and null clears it.
• UI.
  • The editor opens with the stored config, or with the built-in rules if nothing is stored. Validation is deep-strict.
  • Actions are Save, Load built-in rules and Clear configuration.
  • The card says the gateway does not read the config yet.
  • A Current routing card next to the editor draws the rules as a top-to-bottom table: rule number, the includes substrings (joined by "or"), any excludes ("but not", in red), and the provider chain. A final "Any other model → No preference" row shows the fallthrough. The diagram follows the editor contents live. A badge says whether it shows the stored configuration, the built-in rules, or unsaved changes. If the JSON is invalid, the card asks you to fix it instead of drawing a diagram.
  • The Preview a model ID field now sits in that card. It still prints the resolved order, and it also highlights the rule that matches and dims the others. When no rule matches, it highlights the fallthrough row. findPreferredProviderOrderRuleIndex in preferred-provider-order.ts backs the highlight, and resolvePreferredProviderOrder now uses it too.
  • On xl screens the two cards sit side by side; on smaller screens they stack.

Verification

• Manual browser test against a local Postgres as a fake admin: the built-in rules render, previewing anthropic/claude-sonnet-4.5 highlights rule 2, and previewing openai/gpt-oss-120b skips rule 1 (excluded) and highlights the fallthrough row.
• tsgo --noEmit passed for apps/web, @kilocode/db and @kilocode/web-shared (both tsconfigs).
• oxlint and oxfmt are clean on the changed files, and git diff --check passes.
• Jest against a local DB: preferred-provider-order.test.ts (with new cases for the rule index), preferred-provider-order-router.test.ts, auto-free-config-router.test.ts and gateway-model-lists-router.test.ts passed.
• The earlier web-test (5/8) failure was a 5s beforeAll timeout in organization-sales-demo-router.test.ts, which this PR does not touch. That suite passes locally, and the timeout does not appear in any of the 25 most recent failed runs of the workflow, so it was a one-off flake on a slow runner.

Visual Changes

New "Provider Order" tab under Admin → Gateway, with the editor on the left and the current routing on the right.

Preview matches a rule	Preview falls through (gpt-oss is excluded)

Reviewer Notes

• [AISAGE-707] feat(admin): configure custom model pricing in the gateway admin panel #7499 (custom pricing admin) also adds an ai_gateway_config column and touches page.tsx, admin-router.ts, schema-types.ts and the exact-row router tests. Whichever PR merges second has to regenerate its migration.

Follow-up: connect the gateway to the stored config

• Read the column with readDb through a 60s createCachedFetch, the same pattern as getVercelModelIdOverrides and getUnavailableModelIds. Use PREFERRED_PROVIDER_ORDER_FALLBACK_CONFIG when the column is null, and keep the last valid value when the stored config is invalid.
• getPreferredProviderOrder then becomes async. applyPreferredProvider is already called from the async applyProviderSpecificLogic. The catalog pricing loop in openrouter/index.ts should load the config once and call resolvePreferredProviderOrder for each model.
• If [AISAGE-707] perf(ai-gateway): serve gateway config from one Redis snapshot #8107 (one Redis snapshot for gateway config) lands first, read the column from that snapshot instead.

Further configuration suggestions

These are hardcoded next to the provider order today and would fit the same ai_gateway_config pattern:

1. Ignored providers. getIgnoredProviders hardcodes "ignore anthropic for Claude unless non-trial enterprise or BYOK". This could be a second rule list, { includes, excludes, ignore, unlessAnthropicAllowed }, or an ignore field on the same rules.
2. More OpenRouter routing fields for each rule. Each rule could also set allow_fallbacks, only, sort (price / throughput / latency), quantizations or max_price. Then an incident, such as forcing Kimi onto Bedrock only, can be handled without a deploy.
3. Model family matchers. isClaudeModel, isKimiModel, isGlmModel and the others are substring checks spread across provider files, and they also control request transforms. One stored "family → substrings" table would keep provider order, ignores and transforms consistent when a new model name, for example a rebrand, does not match the old substring.
4. Regex or prefix matching. If substrings stop being precise enough, add an optional match: "substring" | "prefix" | "regex" to each rule. Validate regexes on save and cap their length.
5. Per-organization overrides. Enterprise customers sometimes need data-residency or contract-specific provider lists. An optional organization_ids on a rule, which wins over global rules, would cover that without custom code.
6. Vercel provider slug mapping. provider.order also goes to Vercel through openRouterToVercelInferenceProviderId, which uses the hardcoded openRouterToVercelInferenceProviderMapping. That mapping could sit next to the Vercel model ID overrides, so a new provider in the order does not also need a deploy.
   6a. Warn about unknown provider slugs. On save, flag slugs that are not in OpenRouterInferenceProviderIdSchema or in the synced endpoint tags. A warning rather than an error, because the enum can lag. A typo would otherwise drop that preference without any sign.
7. Kilo-exclusive model and custom pricing constants. kilo-exclusive-models.ts and custom-pricing.constants.ts are still code. [AISAGE-707] feat(admin): configure custom model pricing in the gateway admin panel #7499 already starts on pricing.
8. Audit and history. Gateway config edits are now routing changes that take effect in production. Record them through the admin audit log from feat(admin): record admin actions in audit log and Axiom #8142, with the before and after JSON. A "revert to previous version" action would also help, and so would showing who last edited each ai_gateway_config column.
9. Dry run against traffic. Extend the preview so it lists the models from the current catalog whose provider order would change with the unsaved config, before saving.

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Reviewed the incremental commit that adds the ProviderOrderRoutingDiagram and moves the model preview plus rule highlight into a new RoutingCard; the rule-index helper, the resolver refactor, and the new UI are consistent, and the layout change leaves the configuration editor logic untouched.

Assumptions: [INFERENCE] No code was executed: tsgo, oxlint, Jest, and the browser rendering of the new diagram were not run here. Review relied on reading the changed files plus the unchanged consumers (matchesRule, the admin router, Badge variants, and the status-destructive tokens, which all exist). The memory-leak check is clean: the new components add no effects, listeners, timers, or subscriptions. No markdown files changed, so the image-format rule does not apply.

Review Notes

• findPreferredProviderOrderRuleIndex returns -1 for no match, and resolvePreferredProviderOrder indexes with it (rules[-1] is undefined), so it still returns [] with the same copy-per-call behavior.
• The diagram highlight props are consistent with the documented contract: null means no preview, matchedRuleIndex === index highlights a rule, -1 highlights the fallthrough row, and the other rows are dimmed only while previewing.
• RoutingCard memoizes parseConfig(json), so the diagram only renders for valid JSON and otherwise shows the "fix the JSON" message; the preview result and the highlighted rule both come from the same validated config, so they cannot disagree.
• The provider-order-preview input id is now rendered exactly once, since the old ModelPreview was removed, so the label association stays unique.

Files Reviewed (4 files)

• apps/web/src/app/admin/gateway/PreferredProviderOrderContent.tsx
• apps/web/src/app/admin/gateway/ProviderOrderRoutingDiagram.tsx
• packages/web-shared/src/lib/ai-gateway/providers/preferred-provider-order.test.ts
• packages/web-shared/src/lib/ai-gateway/providers/preferred-provider-order.ts

Previous Review Summaries (2 snapshots, latest commit 0d31c16)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 0d31c16)

Status: No Issues Found | Recommendation: Merge

Executive Summary

Reviewed the incremental rewrite that replaces the single substring matcher with ordered includes/excludes rules; the new matchesRule logic, the merged OpenAI rule, the shared uniqueness helper, and the updated editor copy are consistent, and the resolver still returns no preference for gpt-oss models.

Assumptions: [INFERENCE] The 031c166 commit was verified by reading the changed files and the unchanged consumers (getPreferredProviderOrder, applyPreferredProvider, the catalog pricing loop, the admin router, and deepStrict); no code was executed, no DB-backed router tests were run, and no browser check was done. Reasoning about Zod 4 chaining (z.array(...).superRefine(...).min(1).max(...), superRefine returning the same schema instance) relies on the PR's tsgo result because node_modules is not installed here.

Review Notes

• The fallback config merges the old openai and gpt rules and expresses the gpt-oss exception as excludes: ['gpt-oss'], so openai/gpt-oss-120b still resolves to no preference and every other family rule is unchanged.
• A rule whose excludes matches is skipped and matching continues to later rules; this is the intended new behavior, exercised by preferred-provider-order.test.ts and the only semantic difference from the old "stop on empty providers" rule. No current model ID contains both gpt-oss and another family substring, so gateway behavior does not change today.
• uniqueStringArray applies the same duplicate check to includes, excludes, and providers with a nested issue path, and includes/providers now require at least one entry, which matches the resolver and the editor copy.
• Cross-rule duplicate includes lists are no longer rejected (the old duplicate-substring check and its test were removed). Two identical rules make the later one unreachable; this is a configuration-quality nit, not a runtime defect, and the admin sees the rule order.
• PreferredProviderOrderRuleSchema remains a Zod object with refinements, so deepStrict still walks it as an object; the non-strict schema parse in parseConfig continues to enforce the include/exclude conflict.
• The UI only changes descriptive text plus no new subscriptions or listeners, so the memory-leak check is clean. No markdown files are changed, so the image-format rule does not apply.

Files Reviewed (5 incremental + targeted context)

• apps/web/src/app/admin/gateway/PreferredProviderOrderContent.tsx
• apps/web/src/routers/admin/preferred-provider-order-router.test.ts
• packages/db/src/schema-types.ts
• packages/web-shared/src/lib/ai-gateway/providers/preferred-provider-order.test.ts
• packages/web-shared/src/lib/ai-gateway/providers/preferred-provider-order.ts

Previous review (commit 0f3393e)

Status: No Issues Found | Recommendation: Merge

Executive Summary

Reviewed the additive preferred_provider_order migration, the new schema and resolver, the admin tRPC router, and the Provider Order editor; the refactor reproduces the previous getPreferredProviderOrder behavior for all existing model families, and the config is not yet read by the gateway.

Assumptions: [INFERENCE] The 40k-line 0292_snapshot.json is Drizzle-generated and was verified only by diffing it against 0291_snapshot.json (only the new jsonb column and id/prevId change). The new behavior for a model ID containing both gpt-oss and another family substring differs from the old if chain, as the PR already documents, but no current model ID has that shape. DB-backed router tests and a browser check were not run here.

Review Notes

• Migration 0292 is a single nullable ADD COLUMN "preferred_provider_order" jsonb with no default or backfill, so it does not rewrite or lock the singleton table. _journal.json has a contiguous idx 292 entry and the snapshot chain links to 0291's id.
• resolvePreferredProviderOrder plus PREFERRED_PROVIDER_ORDER_FALLBACK_CONFIG matches the removed if chain rule by rule (openai/gpt including the gpt-oss exclusion, claude, minimax, mistral, kimi, step-, deepseek, glm, qwen), and every predicate it replaces is a plain includes check.
• The removed imports in apply-provider-specific-logic.ts are no longer referenced, and the remaining imports are still used.
• admin.preferredProviderOrder.get/set use adminProcedure, scope the update to the one column, and mirror the existing Auto Free config router pattern. Exact-row test assertions that needed the new column were updated.
• The editor mirrors AutoFreeContent (same dual deep-strict plus schema parse, same save/clear/disabled logic); no new listeners or subscriptions were introduced, so no leak path.
• Unknown provider slugs and the eventual read of the stored config remain follow-ups the PR already lists.

Files Reviewed (15 files)

• apps/web/src/app/admin/gateway/PreferredProviderOrderContent.tsx
• apps/web/src/app/admin/gateway/page.tsx
• apps/web/src/routers/admin-router.ts
• apps/web/src/routers/admin/auto-free-config-router.test.ts
• apps/web/src/routers/admin/gateway-model-lists-router.test.ts
• apps/web/src/routers/admin/preferred-provider-order-router.test.ts
• apps/web/src/routers/admin/preferred-provider-order-router.ts
• packages/db/src/migrations/0292_preferred_provider_order.sql
• packages/db/src/migrations/meta/0292_snapshot.json
• packages/db/src/migrations/meta/_journal.json
• packages/db/src/schema-types.ts
• packages/db/src/schema.ts
• packages/web-shared/src/lib/ai-gateway/providers/apply-provider-specific-logic.ts
• packages/web-shared/src/lib/ai-gateway/providers/preferred-provider-order.test.ts
• packages/web-shared/src/lib/ai-gateway/providers/preferred-provider-order.ts

---

_{Reviewed by deepseek-flash-latest · Input: 88.1K · Output: 8.4K · Cached: 447.6K}

_{Review guidance: REVIEW.md from base branch main}