[chrarnoldus]

Summary

Rate-limited free models (today only free Gemma, which free users reach through kilo-auto/small for session titles and similar) no longer count free_model_usage rows and insert one per request. They now count against a Vercel Firewall rule, free-model-inference, through @vercel/firewall's checkRateLimit. The rule keeps the existing limit of 200 requests per hour per key.

• Key selection is unchanged: server-side products (cloud-agent, code-review, app-builder, gastown) arriving from Cloudflare IPs are keyed user:<id> and still need auth. Everything else is keyed ip:<address>.
• Shared helper: the header minimisation and throttled missing-rule Sentry report are moved out of gateway-account-rate-limit.ts into firewall-rate-limit.ts, and both rules use it. The throttle is now per rule, so one missing rule can't hide another.
• Promotion limit removed: the anonymous 10,000/24h limit could never fire, because the 200/h per-IP limit caps an IP at about 4,800 per 24h. Its PROMOTION_MODEL_LIMIT_REACHED response and the promotion_limit_reached error type are removed too.
• Admin tooling removed: the /admin/free-model-usage page, the free and promoted model usage stats routes, the admin.freeModelUsage tRPC router (including "rate limit my IP"), and the related constants. With no new writes, they would only show stale data.
• Kept: the free_model_usage table, its schema, the FK nulling on user deletion, and the cleanup cron, which drains the existing rows before the table is dropped in a follow-up.

Required before deploy: create the firewall rule

checkRateLimit treats an unknown rule ID as "not limited". Until the rule exists, the free-model limit is off and Sentry gets Firewall rate limit 'free-model-inference' is not configured (throttled to once a minute per instance, tag source:free_model_inference). Publishing the rule before this deploys is harmless, because nothing calls it yet.

Create the rule in each Vercel project that serves the LLM proxy. Rules and counters are separate per project.

Project	Proxy routes
kilocode-ai-gateway	/api/v1
kilocode-app	/api/openrouter, /api/gateway
kilocode-global-app	same as kilocode-app, only if it still deploys

In each project:

1. Firewall → Configure → + New Rule, named e.g. Free model inference.
2. If: @vercel/firewall, Rate limit ID free-model-inference (exact match).
3. Then: Rate Limit, Fixed Window, 3600s (1 hour), 200 requests, action Default (429). Leave the counting key alone; the code supplies ip:<address> or user:<id>, which replaces the IP key.
4. Save Rule → Review Changes → Publish.

To trial it first, set the action to Log, watch the rule's traffic on the Firewall overview, then switch to 429. The limit can be tuned in the dashboard without a deploy.

How this differs from the old database limit

The number and window are the same, but the behaviour is not identical:

• Plan: a 1-hour window needs Enterprise. Hobby and Pro cap the window at 10 minutes, where the closest setting would be 600s / 33 requests.
• Fixed window, not sliding: the old limit counted the last 60 minutes. A fixed window resets on the hour, so a caller can send up to about 400 requests across a window boundary. Enterprise also offers Token Bucket, which smooths this out.
• Counted at the check: every request that reaches the check counts, including ones later rejected by auth or validation. The old insert happened after those checks.
• Per region and per project: counters are kept per region, and kilocode-ai-gateway runs in fra1 and sfo1. A key also gets a separate allowance in each project. The old database count was global, so the real ceiling is somewhat higher.

The existing account cap needs its gateway-inference rule in the same projects. A missing rule shows up in Sentry as the same "not configured" message for gateway-inference.

Verification

Re-run after merging the latest main (4e8bca43c):

• Jest, run without the shared Postgres setup (these suites mock the DB): firewall-rate-limit, free-model-rate-limit, gateway-account-rate-limit, llm-proxy, llm-proxy-helpers, typesafe/handler, models, feature-detection. 8 suites, 386 tests passed. New llm-proxy cases cover the per-IP key, the per-user key, and the auth requirement for server-side products on Cloudflare IPs.
• tsgo typecheck passes for apps/web, apps/ai-gateway and packages/web-shared, and the packages/trpc build passes.
• On the original change: oxlint on changed paths, oxfmt --list-different, and git diff --check were clean.
• Not run: the full DB-backed Jest suite (no Postgres in this environment) and a deployed check against a real firewall rule.

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Reviewed the free-model rate-limit migration from DB-backed counting to the shared Vercel Firewall helper; the refactor is internally consistent, all removed symbols have no remaining references, and the module-level Sentry throttle Map is bounded by fixed rule IDs (no leak).

Files Reviewed (20 files)

• apps/web/src/app/admin/api/free-model-usage/stats/route.ts (deleted)
• apps/web/src/app/admin/api/promoted-model-usage/stats/route.ts (deleted)
• apps/web/src/app/admin/components/AppSidebar.tsx
• apps/web/src/app/admin/components/FreeModelUsageStats.tsx (deleted)
• apps/web/src/app/admin/components/PromotedModelUsageStats.tsx (deleted)
• apps/web/src/app/admin/components/RateLimitTesting.tsx (deleted)
• apps/web/src/app/admin/free-model-usage/page.tsx (deleted)
• apps/web/src/routers/admin-router.ts
• apps/web/src/routers/admin/free-model-usage-router.ts (deleted)
• packages/web-shared/src/lib/ai-gateway/firewall-rate-limit.test.ts
• packages/web-shared/src/lib/ai-gateway/firewall-rate-limit.ts
• packages/web-shared/src/lib/ai-gateway/free-model-rate-limit.test.ts
• packages/web-shared/src/lib/ai-gateway/free-model-rate-limit.ts
• packages/web-shared/src/lib/ai-gateway/gateway-account-rate-limit.ts
• packages/web-shared/src/lib/ai-gateway/handlers/embeddings.ts
• packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.test.ts
• packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.ts
• packages/web-shared/src/lib/constants.ts
• packages/web-shared/src/lib/free-model-rate-limiter.ts (deleted)
• packages/web-shared/src/lib/proxy-error-types.ts

Verified: no dangling references to removed exports (checkFreeModelRateLimit, checkFreeModelRateLimitByUser, checkPromotionLimit, logFreeModelRequest, adminFreeModelUsageRouter, promotion_limit_reached, removed constants, admin components); firewall-rate-limit per-rule throttle Map is bounded; rate-limit key derivation and per-user/per-IP selection in resolveFreeModelRateLimitKey preserve the previous behavior and are covered by new tests.

---

_{Reviewed by deepseek-v4.1-flash · Input: 87.3K · Output: 16.9K · Cached: 1.4M}

_{Review guidance: REVIEW.md from base branch main}