[chrarnoldus]

Summary

This adds a per-connection rate limit to the "Sign in with ChatGPT" route (openai-chatgpt provider). It works like the per-account free-model limit from #8456.

Why. Every ChatGPT connection shares the partner project's OpenAI rate limits. A single connection driven by an automated relay can use them up, and then every other person on the route gets OpenAI rate-limit errors.

How it works:

• openai-chatgpt/rate-limit.ts counts the request against the Vercel Firewall rule openai-chatgpt-inference through the shared isFirewallRateLimited helper, the same way free-model-rate-limit.ts does.
• The bucket key is the connection owner, so each ChatGPT plan gets its own allowance:
  • <kiloUserId>:personal for a personal connection
  • <kiloUserId>:<orgId> for an organization member's connection
  • shared-services:<orgId> for an organization's shared-services connection
• Because the key is the connection, rotating source IP addresses does not raise the allowance.
• handleLlmProxyRequest runs the check right after getProvider, and only when the selected provider carries a ChatGPT connection owner. Other providers are never counted.
• A connection over its limit gets a 429 rate_limit_exceeded with x-should-retry: false, like the free-model limit.
• An over-limit request is not silently served through Kilo credits instead. That keeps the existing rule that the ChatGPT route never quietly switches billing path.
• A log line records the connection key, model and feature, like the free-model limit's log line.

Tracked in AISAGE-759.

Deployment

Create a Vercel Firewall rule on kilocode-ai-gateway before this takes effect:

• Condition: @vercel/firewall, rate limit ID openai-chatgpt-inference.
• Rate limit: fixed window, 1 hour, 1,000 requests. On a plan limited to 10-minute windows, use 10 minutes, 400 requests.
• Key: set in code (rateLimitKey = the connection owner). The dashboard key choice does not matter.
• Action: default (429).

Before the rule exists: checkRateLimit treats an unknown rule as not rate limited, so this change limits nothing. isFirewallRateLimited also reports the missing rule to Sentry (source: openai_chatgpt_inference), throttled to once a minute per instance.

Why these values. In the last 7 days, the busiest IP using the genuine Kilo client (Kilo-Code/… ai-sdk/…) on this route sent at most 871 requests in an hour and 364 in 10 minutes. Several users can share one IP, so these are upper bounds per connection. Neither limit would have refused a genuine Kilo request in the past day. The detailed figures are in AISAGE-759.

Per-region counters. Vercel counts each region separately, so a connection that is served by both fra1 and sfo1 can get up to twice the limit.

Retry headers. The refusal sends x-should-retry: false, like the free-model limit. With a window of 10 minutes or longer, an SDK's automatic retries (seconds of backoff) all fail anyway and only spend more of the allowance. Vercel does not report when a window resets, so no accurate retry-after is available. The code comments now require the window to stay at 10 minutes or longer.

Verification

• Ran the new openai-chatgpt/rate-limit.test.ts. It checks the rule id and the bucket keys for personal, organization-member and shared-services connections.
• Added llm-proxy.test.ts cases for three paths:
  • a connection over its limit gets a 429 with x-should-retry: false and makes no upstream call;
  • a connection under its limit is served;
  • other providers are never counted.
• Ran all openai-chatgpt/, handlers/, typesafe/ and free-model rate-limit suites; all pass.
• Ran pnpm --filter @kilocode/web-shared typecheck and oxlint on the changed files; both are clean.
• No manual test against the Vercel Firewall: the rule does not exist yet, and checkRateLimit only works on deployed Vercel infrastructure.

Visual Changes

N/A

Reviewer Notes

• Placement of the check. It runs after getProvider, because the connection owner is only known once the ChatGPT route has been chosen. Choosing the route includes resolving the delegated token, so an over-limit request still pays for that lookup. A cheaper check before routing would have to key on the account instead of the connection.
• Interaction with the account-wide limit. The existing gateway-inference limit, keyed on the account, still applies first. This rule is a tighter, connection-scoped bucket on top of it.

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Reviewed the incremental commit 9ea78599c, which only rewrites two explanatory comments so the x-should-retry: false refusal is tied to the firewall window being ten minutes or longer; both comments match the constant they describe and contain no code changes.

Assumptions: [INFERENCE] The openai-chatgpt-inference firewall rule and its window are configured in kilocode-ai-gateway and cannot be verified from the repository; the new comment states the requirement but does not enforce it.

Files Reviewed (2 files)

• packages/web-shared/src/lib/ai-gateway/llm-proxy-helpers.ts
• packages/web-shared/src/lib/ai-gateway/openai-chatgpt/rate-limit.ts

Previous Review Summary (commit 9ee7af8)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 9ee7af8)

Status: No Issues Found | Recommendation: Merge

Executive Summary

Reviewed the per-connection ChatGPT rate limit in the five changed files: the check runs on the only code path that can select an openai-chatgpt provider, the bucket keys for personal, organization-member, and shared-services connections are distinct, and the tests cover the over-limit, under-limit, and non-ChatGPT-provider paths.

Assumptions: [INFERENCE] The openai-chatgpt-inference firewall rule, its window, and its limit are configured in the kilocode-ai-gateway Vercel project; this cannot be verified from the repository, and until the rule exists the check only reports a missing rule to Sentry and limits nothing. Buckets are keyed per connection rather than per ChatGPT plan, so one plan connected to several owners (the store documents that this is possible) holds several allowances.

Files Reviewed (5 files)

• packages/web-shared/src/lib/ai-gateway/openai-chatgpt/rate-limit.ts
• packages/web-shared/src/lib/ai-gateway/openai-chatgpt/rate-limit.test.ts
• packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.ts
• packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.test.ts
• packages/web-shared/src/lib/ai-gateway/llm-proxy-helpers.ts

---

_{Reviewed by deepseek-flash-latest · Input: 22.8K · Output: 1.4K · Cached: 154.1K}

_{Review guidance: REVIEW.md from base branch main}