Skip to content

refactor(ai-gateway): return every inference provider for Kilo-exclusive models - #7772

Closed
chrarnoldus wants to merge 1 commit into
mainfrom
kilo/inference-providers
Closed

chrarnoldus wants to merge 1 commit into
mainfrom
kilo/inference-providers

Conversation

@chrarnoldus

@chrarnoldus chrarnoldus commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary

getInferenceProvider returned a single provider. When a Kilo-exclusive model had more than one entry in inference_provider_restriction, it returned null, so such a model was:

  • missing from the models-by-provider snapshot
  • listed without inference_provider_ids in the catalog
  • skipped by the request-level provider.only / provider.ignore check

It is now getInferenceProviders, which returns every distinct provider that may serve the model:

  • the stealth provider for stealth models
  • every restriction entry, deduplicated, when there is a restriction
  • the direct upstream for non-gateway providers
  • an empty list for unrestricted OpenRouter/Vercel models, where the snapshot decides

Callers updated:

  • Snapshot sync: lists the model under each provider.
  • Catalog: publishes every provider in inference_provider_ids.
  • checkExclusiveModelProviderAllowed: allows the request when any provider survives only/ignore, and names every provider in the 403.

Every current exclusive model has zero or one restriction entry, so today's behavior does not change. This is a prep PR for #7291, where public custom LLMs declare several inference providers.

Verification

  • No manual testing; this is a refactor with no behavior change for the current model list.

Visual Changes

N/A

Reviewer Notes

Unit tests cover multi-provider allow/deny in checkExclusiveModelProviderAllowed, plus multi-provider and repeated restriction entries in getInferenceProviders.

@chrarnoldus chrarnoldus self-assigned this Oct 16, 2026
@kilo-code-bot

kilo-code-bot Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

The getInferenceProvider → getInferenceProviders refactor is behavior-preserving for every current exclusive model (all have zero or one restriction entry), the prior deduplication and test-cleanup findings are resolved, and no new correctness, security, or memory-leak issues were found in the rebased diff.

Files Reviewed (8 files)
  • apps/web/src/lib/providers/openrouter/sync-providers.ts
  • packages/web-shared/src/lib/ai-gateway/llm-proxy-helpers.ts
  • packages/web-shared/src/lib/ai-gateway/llm-proxy-helpers.test.ts
  • packages/web-shared/src/lib/ai-gateway/models.test.ts
  • packages/web-shared/src/lib/ai-gateway/providers/kilo-exclusive-model.ts
  • packages/web-shared/src/lib/ai-gateway/providers/kilo-exclusive-model.test.ts
  • packages/web-shared/src/lib/ai-gateway/providers/openrouter/index.ts
  • packages/web-shared/src/lib/ai-gateway/providers/openrouter/index.test.ts
Previous Review Summaries (2 snapshots, latest commit cba96eb)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit cba96eb)

Status: No Issues Found | Recommendation: Merge

Executive Summary

The incremental changes resolve both prior suggestions: getInferenceProviders now derives providers from a Set (deduplicating repeated restriction entries) and the test cleanup only splices when indexOf finds the model; no new issues found on the changed lines.

Files Reviewed (3 files)
  • packages/web-shared/src/lib/ai-gateway/providers/kilo-exclusive-model.ts
  • packages/web-shared/src/lib/ai-gateway/providers/kilo-exclusive-model.test.ts
  • packages/web-shared/src/lib/ai-gateway/llm-proxy-helpers.test.ts

Previous review (commit fae4971)

Status: 2 Issues Found | Recommendation: Address before merge

Executive Summary

The getInferenceProvider → getInferenceProviders refactor is behavior-preserving for all current model data and introduces no production memory leak; the only findings are two non-blocking hardening suggestions (undeduplicated restriction slugs, and an unguarded test cleanup splice).

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 2
Issue Details (click to expand)

SUGGESTION

File Line Issue
packages/web-shared/src/lib/ai-gateway/providers/kilo-exclusive-model.ts 215 getInferenceProviders does not dedupe inference_provider_restriction; a repeated slug would repeat in the 403 message and can duplicate entries in sync-providers.
packages/web-shared/src/lib/ai-gateway/llm-proxy-helpers.test.ts 189 splice(indexOf(...), 1) removes the last array element when indexOf returns -1; guard with >= 0 like the sibling tests.
Files Reviewed (8 files)
  • apps/web/src/lib/providers/openrouter/sync-providers.ts - no issues
  • packages/web-shared/src/lib/ai-gateway/providers/kilo-exclusive-model.ts - 1 issue
  • packages/web-shared/src/lib/ai-gateway/providers/kilo-exclusive-model.test.ts - no issues
  • packages/web-shared/src/lib/ai-gateway/llm-proxy-helpers.ts - no issues
  • packages/web-shared/src/lib/ai-gateway/llm-proxy-helpers.test.ts - 1 issue
  • packages/web-shared/src/lib/ai-gateway/models.test.ts - no issues
  • packages/web-shared/src/lib/ai-gateway/providers/openrouter/index.ts - no issues
  • packages/web-shared/src/lib/ai-gateway/providers/openrouter/index.test.ts - no issues

Fix these issues in Kilo Cloud


Reviewed by deepseek-v4.1-flash-1017 · Input: 58.9K · Output: 12.1K · Cached: 1.1M

Review guidance: REVIEW.md from base branch main

@chrarnoldus
chrarnoldus force-pushed the kilo/inference-providers branch from cba96eb to 8fc4624 Compare October 19, 2026 12:27

Copy link
Copy Markdown
Contributor Author

Closing: #8091 already landed the same change on main (getInferenceProviders returning every provider, multi-provider checkExclusiveModelProviderAllowed, snapshot and catalog updates), so this PR has nothing left to add. #7291 now builds on main directly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant