Skip to content

[AISAGE-707] refactor(gateway): replace Kilo-exclusive stealth flag with inference_providers - #8091

Merged
chrarnoldus merged 2 commits into
mainfrom
refactor/kilo-exclusive-inference-providers
Oct 19, 2026
Merged

chrarnoldus merged 2 commits into
mainfrom
refactor/kilo-exclusive-inference-providers

Conversation

@chrarnoldus

@chrarnoldus chrarnoldus commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Jira: AISAGE-707

Summary

  • Renames KiloExclusiveModel.inference_provider_restriction to inference_providers, and getKiloExclusiveInferenceProviderRestriction to getKiloExclusiveInferenceProviders.
  • inference_providers may only be empty for models served through OpenRouter or Vercel AI Gateway, where empty means unrestricted. A test enforces this for every Kilo-exclusive model.
  • Removes the stealth model flag. A model is a stealth model when its inference_providers contains stealth (isStealthModel, isKiloStealthModel).
  • Existing models: the Martian stealth models and Glyph Cluster use ['stealth'], Reflection Beam keeps ['reflection'], and free Gemma stays [] (OpenRouter).

Behavior notes

  • applyKiloExclusiveModelSettings only adds provider.only for OpenRouter/Vercel models whose providers are not stealth. stealth is a public identity that hides the real upstream, not a routing target, so Glyph Cluster is still routed through Vercel without a provider restriction, as it was before.
  • getInferenceProvider is replaced by getInferenceProviders, which returns one entry per listed provider and no longer derives a slug from the direct provider ID. Before, models with several providers got null and were silently skipped:
    • checkExclusiveModelProviderAllowed now allows the request when at least one listed provider passes the caller's provider.only/ignore. Otherwise it returns the 403 naming all listed providers.
    • Provider sync adds an exclusive model under every listed provider.
    • Every current model lists at most one provider, so behavior is unchanged today.
  • The model catalog's inference_provider_ids for exclusives now comes directly from inference_providers. The result is unchanged.
  • Organization model access and provider allow lists now resolve stealth models to {stealth} without a catalog lookup. The catalog already indexed these models under stealth.
  • Free-endpoint data policy now marks only the stealth endpoint for stealth exclusives. Before, it matched any provider with the same normalized slug. Stealth slugs are stealth/..., so in practice nothing changes.

Verification

  • pnpm --filter @kilocode/web-shared typecheck, pnpm --filter web typecheck, pnpm --filter ai-gateway typecheck
  • oxlint on changed files
  • Jest (apps/web config) on the latest commit: packages/web-shared/src/lib/ai-gateway (includes the new checkExclusiveModelProviderAllowed tests), packages/web-shared/src/lib/model-allow.server.test.ts, packages/web-shared/src/lib/organizations, apps/web/src/lib/providers/openrouter, apps/web/src/lib/model-api-kinds.test.ts, apps/web/src/app/admin/api/auto-routing. 128 suites passed (2422 tests, 2 skipped).

@chrarnoldus chrarnoldus self-assigned this Oct 19, 2026
@chrarnoldus chrarnoldus changed the title refactor(gateway): replace Kilo-exclusive stealth flag with inference_providers [AISAGE-707] refactor(gateway): replace Kilo-exclusive stealth flag with inference_providers Oct 19, 2026
@kilo-code-bot

kilo-code-bot Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Incremental review of commit 9befbb7 confirms getInferenceProviders and the any-provider eligibility check preserve current behavior for every existing Kilo-exclusive model while correctly supporting multiple providers.

Verification Notes

  • getInferenceProviders returns one entry per listed provider; the only empty case is gemma_4_26b_a4b_it_free_model (OpenRouter, unrestricted), which was also skipped by the previous getInferenceProvider logic.
  • checkExclusiveModelProviderAllowed now passes when any listed provider satisfies only/ignore, and the 403 message names all providers. Single-provider models (all current exclusives) resolve identically to before.
  • sync-providers now attaches a multi-provider exclusive to each matching provider; per-provider dedup by slug in normalizedProviders prevents duplicate entries, and each provider receives its own convertFromKiloExclusiveModel object.
  • No remaining references to getInferenceProvider, inference_provider_restriction, or getKiloExclusiveInferenceProviderRestriction.
  • No new memory-retention or unbounded-growth paths: the only mutation is the test-local kiloExclusiveModels.push/splice pair, which is balanced in beforeEach/afterEach.
Files Reviewed (7 files)
  • apps/web/src/lib/providers/openrouter/sync-providers.ts
  • packages/web-shared/src/lib/ai-gateway/llm-proxy-helpers.ts
  • packages/web-shared/src/lib/ai-gateway/llm-proxy-helpers.test.ts
  • packages/web-shared/src/lib/ai-gateway/models.test.ts
  • packages/web-shared/src/lib/ai-gateway/providers/kilo-exclusive-model.ts
  • packages/web-shared/src/lib/ai-gateway/providers/kilo-exclusive-model.test.ts
  • packages/web-shared/src/lib/ai-gateway/providers/openrouter/index.test.ts
Previous Review Summary (commit e9c9e36)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit e9c9e36)

Status: No Issues Found | Recommendation: Merge

Files Reviewed (16 files)
  • apps/web/src/app/admin/api/auto-routing/benchmark-config/route.test.ts
  • apps/web/src/lib/model-api-kinds.test.ts
  • apps/web/src/lib/providers/openrouter/free-endpoint-data-policy.test.ts
  • apps/web/src/lib/providers/openrouter/free-endpoint-data-policy.ts
  • packages/web-shared/src/lib/ai-gateway/kilo-exclusive-models.test.ts
  • packages/web-shared/src/lib/ai-gateway/kilo-exclusive-models.ts
  • packages/web-shared/src/lib/ai-gateway/models.test.ts
  • packages/web-shared/src/lib/ai-gateway/openai-chatgpt/routing.test.ts
  • packages/web-shared/src/lib/ai-gateway/providers/apply-provider-specific-logic.test.ts
  • packages/web-shared/src/lib/ai-gateway/providers/kilo-exclusive-model.test.ts
  • packages/web-shared/src/lib/ai-gateway/providers/kilo-exclusive-model.ts
  • packages/web-shared/src/lib/ai-gateway/providers/openrouter/index.ts
  • packages/web-shared/src/lib/model-allow.server.test.ts
  • packages/web-shared/src/lib/model-allow.server.ts
  • packages/web-shared/src/lib/organizations/effective-model-access.server.test.ts
  • packages/web-shared/src/lib/organizations/group-policies/model-access/model-access.server.ts

Verified the rename is complete with no remaining inference_provider_restriction or 'stealth' flag references, and traced every caller of getInferenceProvider/getKiloExclusiveInferenceProviders: routing, catalog inference_provider_ids, provider sync, free-endpoint data policy, and organization model access all resolve to the same providers as before for every current exclusive model. The stealth early-return in applyKiloExclusiveModelSettings correctly prevents a bogus provider.only: ['stealth'] route, and no new memory-retention or unbounded-growth paths were introduced.


Reviewed by deepseek-v4.1-flash-1017 · Input: 77.6K · Output: 9.2K · Cached: 532.2K

Review guidance: REVIEW.md from base branch main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants