[chrarnoldus]

Summary

The gateway passes request.signal to the upstream fetch, so when the client disconnects, the upstream stream and the usage clone are aborted together. Usage normally arrives in the final chunk, so the request is then recorded with 0 tokens. OpenRouter and Vercel recover the cost through the generation lookup. Providers without a lookup (Martian, which serves the stealth models) were billed $0, including when the client stopped reading after it already had the full response.

The upstream is now aborted on client disconnect only when the cost stays recoverable:

• the provider has a generation lookup (openrouter, vercel), or
• the request uses the user's own key (userByok), so Kilo does not pay.

For every other provider, the upstream keeps streaming after the client leaves, bounded by the existing upstream duration budget, and countAndStoreUsage (inside after()) records the real usage.

Verification

• Local stack with the real Martian upstream (stealth/claude-sonnet-4.6). Before: aborting mid-stream on /chat/completions, or at content_block_stop on /messages, recorded 0 tokens and $0. After: both recorded full input and output usage and the matching cost.
• An OpenRouter-routed model still aborts the upstream on disconnect and is billed through the generation lookup.

Visual Changes

N/A

Reviewer Notes

• On Vercel, request.signal only fires on client disconnect when the function sets supportsCancellation, which kilocode-ai-gateway does not. Production therefore already keeps reading in most cases; the abort reproduces under next dev. This change keeps billing correct if cancellation is enabled later. With cancellation enabled, Vercel terminates work outside after()/waitUntil, and usage accounting already runs in after().
• Trade-off: for providers without a lookup, a cancelled request now costs the full generation. That is what the upstream bills anyway.
• The disconnect log no longer claims that upstream work is always aborted.
• Companion PR: [AISAGE-757] fix(ai-gateway): bill message_start input tokens on early stream end #8551 records message_start input tokens on the Messages API.

[kilo-code-bot]

Code review findings for 7d9fbf1

[kilo-code-bot]

WARNING: userByok does not cover every provider the user pays for

The stated rule is to abort on disconnect when the user's own key pays, but userByok is null for two user-paid providers:

• custom (packages/web-shared/src/lib/ai-gateway/providers/get-provider.ts:175) is served with the organization's stored custom-LLM credentials.
• openai-chatgpt (packages/web-shared/src/lib/ai-gateway/openai-chatgpt/routing.ts:328) is served on the user's delegated ChatGPT connection.

Both hit hasGenerationLookup(...) === false and userByok === null, so after this change their upstream is no longer aborted on client disconnect and keeps generating (up to the 10-minute upstream budget) into a response nobody receives. Kilo does not pay, but the user's own ChatGPT allowance or custom-LLM key does, and the request-log/usage path records no benefit for them. Consider basing the decision on who pays rather than on userByok alone (for example also aborting for openai-chatgpt and custom).

[kilo-code-bot]

SUGGESTION: This comment is no longer accurate for the non-aborting path

When abortUpstreamOnClientDisconnect is false the request signal is not forwarded, so upstream-request.ts no longer sets clientDisconnected and a genuine UPSTREAM_DURATION_BUDGET_MS timeout after the client left is logged and reported as an upstream failure rather than being suppressed. The claim that any later abort "is a consequence of the client going away" now holds only for the providers that still receive the signal; it would be worth qualifying it so incident triage does not dismiss a real budget timeout.

[kilo-code-bot]

Code Review Summary

Status: 2 Issues Found | Recommendation: Address before merge

Executive Summary

The change correctly keeps billing usage for providers without a generation lookup, but the userByok-based abort condition misses the custom and openai-chatgpt providers, where the user's own credentials pay, so those requests now keep generating after the client disconnects.

Assumptions: [INFERENCE] With the upstream no longer aborted for lookup-less providers, the abandoned client-facing branch of the response.clone() tee (the rewrite pipeline's cancel() does not cancel its reader, and controller.enqueue throws after cancel) is no longer drained and can buffer the remainder of the stream for up to the 10-minute upstream budget. This is bounded per invocation, but I could not verify its practical size without running the stack.

Overview

Severity	Count
CRITICAL	0
WARNING	1
SUGGESTION	1

Issue Details (click to expand)

WARNING

File	Line	Issue
packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.ts	997	userByok is null for custom and openai-chatgpt, which run on the user's own credentials, so their upstream is no longer aborted on disconnect and keeps consuming the user's quota

SUGGESTION

File	Line	Issue
packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.ts	363	Comment still claims every later abort is caused by the client going away; without a forwarded signal a real upstream budget timeout is now reported

Files Reviewed (3 files)

• packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.ts - 2 issues
• packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.test.ts - no issues
• packages/web-shared/src/lib/ai-gateway/processUsage.ts - no issues (refactor into generationLookupProviderFor preserves existing lookup behavior)

Fix these issues in Kilo Cloud

---

_{Reviewed by deepseek-flash-latest · Input: 169.3K · Output: 20.3K · Cached: 1.6M}

_{Review guidance: REVIEW.md from base branch main}

[chrarnoldus]

Closing: request cancellation (supportsCancellation) is off in production for the AI gateway, so a client disconnect does not abort the upstream there. A production test confirmed that aborted streams are read to completion and billed in full. This change would only matter if cancellation were enabled later.