[chrarnoldus]

Summary

The free-model-inference Vercel Firewall rate limit applied only to Kilo-exclusive models flagged rate-limited (just free Gemma 4 26B). It now applies to every free model.

• llm-proxy gates the check on isFreeModelExcludingCustomLlms instead of isKiloExclusiveRateLimitedModel. This covers :free models, openrouter/free, free stealth alpha models, unpriced Kilo-exclusive models, and models that kilo-auto/free resolves to.
• Custom LLMs stay exempt. They count as "free" only because Kilo does not bill them, and they are limited to the organizations that configure them.
• Removed the now-unused rate-limited flag and isKiloExclusiveRateLimitedModel.

Background

The limit used to be broader. Until #4994 (fix(ai-gateway): scope model rate limits by flag, Aug 4) it covered every enabled Kilo-exclusive free model, plus kilo-auto/free (#2945) and experimented models. #4994 narrowed it to models flagged rate-limited, which was only free Gemma. This PR goes further than the pre-#4994 scope: it also covers OpenRouter :free models, which were never rate limited.

Behavior changes to note

• All free models share one firewall bucket per key (ip:<ip>, or user:<id> for server-side products on Cloudflare IPs). Check that the firewall rule's window and limit fit the higher combined free-model traffic before merging.
• Requests for any free model from server-side products (cloud-agent, code-review, app-builder, gastown) on Cloudflare IPs now need authentication (401 otherwise). Before, only Gemma did.

Verification

• jest for llm-proxy.test.ts, models.test.ts, free-model-rate-limit.test.ts: 171 passed
• pnpm --filter @kilocode/web-shared typecheck: passed
• oxlint and oxfmt on changed files: clean

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

The incremental change at f8555719f restores isFreeModel as the single free-model source of truth and keeps the catalog-backed custom-LLM exemption in the rate-limit gate, so the previously reported :free-suffixed custom LLM case remains fixed; no new issues found in the changed lines.

Verification Notes

• isFreeRequestedModel = await isFreeModel(id) is behaviorally equivalent to the prior isCustomLlm(id) || isFreeModelExcludingCustomLlms(id), since isFreeModel in is-free-model.ts is defined the same way; anonymous-access handling at lines 655/902 is unchanged.
• The gate is now isFreeRequestedModel && !(await isCustomLlm(id)), so catalog custom LLMs stay exempt regardless of id, matching the updated comment and the existing 'acme/custom-model:free' test case.
• isCustomLlm is now evaluated twice on free-model requests (inside isFreeModel and in the gate). Both reads hit the shared 60s cached catalog, so this is a cached-lookup duplicate, not an extra database round trip or correctness risk.
• Removed rate-limited flag and isKiloExclusiveRateLimitedModel have no remaining references in the repo.
• Still not verifiable from the repo: the free-model-inference firewall window/limit for the now-shared free-model bucket, which the PR description asks to re-check before merging.

Files Reviewed (2 files)

• packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.ts - no issues
• packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.test.ts - no issues

Previous Review Summaries (2 snapshots, latest commit afc9896)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit afc9896)

Status: No Issues Found | Recommendation: Merge

Executive Summary

The incremental change at afc98968 resolves the prior heuristic-exemption finding by resolving isCustomLlm once and exempting catalog custom LLMs from the free-model rate limit regardless of their public_id; isFreeRequestedModel keeps its prior isFreeModel semantics, and the added :free-suffixed custom LLM test covers the reported case.

Verification Notes

• isFreeRequestedModel = isCustomLlmRequestedModel || isRateLimitedFreeModel is equivalent to the previous await isFreeModel(id) (isCustomLlm(id) || isFreeModelExcludingCustomLlms(id)), so anonymous-access/auth behavior is unchanged.
• The rate-limit gate is now !isCustomLlmRequestedModel && isFreeModelExcludingCustomLlms(id); isCustomLlm lowercases the id internally and reads the 60s-cached catalog with readDb, so no extra database round trip per request.
• Prior finding on llm-proxy.ts no longer applies: the exemption no longer relies on id heuristics.
• Still not verifiable from the repo: the free-model-inference firewall window/limit for the now-shared free-model bucket, which the PR description asks to re-check before merging.

Files Reviewed (2 files)

• packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.ts - no issues
• packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.test.ts - no issues

Previous review (commit 3bd8e58)

Status: 1 Issue Found | Recommendation: Address before merge

Executive Summary

The gate switch to isFreeModelExcludingCustomLlms is sound, but the claimed custom-LLM exemption is only heuristic: a custom LLM whose public_id matches a free-model pattern is still counted against the shared free-model firewall bucket.

Overview

Severity	Count
CRITICAL	0
WARNING	0
SUGGESTION	1

Issue Details (click to expand)

SUGGESTION

File	Line	Issue
packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.ts	608	Custom LLM exclusion relies on id heuristics, so an org-scoped custom LLM whose public_id ends with :free (or matches openrouter/free / stealth/*-alpha) is still rate-limited despite the exemption stated in the comment and PR description.

Files Reviewed (6 files)

• packages/web-shared/src/lib/ai-gateway/free-model-rate-limit.ts - no issues
• packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.ts - 1 issue
• packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.test.ts - no issues
• packages/web-shared/src/lib/ai-gateway/kilo-exclusive-models.ts - no issues
• packages/web-shared/src/lib/ai-gateway/models.test.ts - no issues
• packages/web-shared/src/lib/ai-gateway/providers/kilo-exclusive-model.ts - no issues

Notes: no remaining references to the removed rate-limited flag or isKiloExclusiveRateLimitedModel were found, and the anchor ordering (rate limit after auto-model resolution, before model-existence and auth checks) matches the stated intent. Not verifiable from the repo: the free-model-inference firewall window/limit that the PR description asks to re-check before merging.

Fix these issues in Kilo Cloud

---

_{Reviewed by deepseek-v4.1-flash-1017 · Input: 35.9K · Output: 3.4K · Cached: 353.8K}

_{Review guidance: REVIEW.md from base branch main}