Skip to content

[AISAGE-756] fix(ai-gateway): strip paid server features from free requests - #8553

Merged
chrarnoldus merged 6 commits into
mainfrom
fix/ai-gateway-free-paid-server-features
Oct 22, 2026
Merged

chrarnoldus merged 6 commits into
mainfrom
fix/ai-gateway-free-paid-server-features

Conversation

@chrarnoldus

@chrarnoldus chrarnoldus commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Free models could trigger paid OpenRouter and Vercel features. plugins (web search, PDF parsing with OCR) and server tools in tools passed through applyProviderSpecificLogic unchanged, including for anonymous callers. OpenRouter bills those features to the gateway account even when the model is free, and usage accounting zeroes the cost of free requests.

applyProviderSpecificLogic now calls removePaidServerFeatures for free models, before any other transform. Anonymous callers can only use free models, so this covers them too:

  • Keeps only plugins OpenRouter does not bill separately (response-healing, context-compression) and removes every other plugin id, so new plugins are removed by default.
  • On OpenRouter, adds { id: 'file-parser', pdf: { engine: 'cloudflare-ai' } }. Without an engine, OpenRouter parses PDFs with mistral-ocr (billed per page) for models that do not accept files natively, so removing the plugin alone would not stop OCR charges. cloudflare-ai is the free engine.
  • Removes every tool the client does not run itself. Only tools without a type, or with type function or custom, are kept. This covers openrouter:* tools and their aliases, such as web_search, web_search_preview, web_search_20250305, web_fetch_*, image_generation, shell and advisor_*. It also covers provider-hosted tools such as Responses file_search and code_interpreter. The rule keeps only known client-run tool types, so new aliases are removed without a code change.
  • Removes tools if no tool is left. Removes tool_choice if no tool is left, or if it forces a removed tool type, so the upstream does not reject the request.
  • Logs a warning with the removed plugins and tool types.

Paid requests from signed-in users, custom LLMs and direct BYOK keep these fields unchanged.

Verification

  • Unit tests for chat completions, Responses and Messages requests. They cover free and paid models, free plugins (including a disabled context-compression), the forced PDF parser on OpenRouter only, tool lists with only server tools, and a forced tool_choice.
  • No manual test against OpenRouter. Server tools and OCR bill per use, and the change only edits the request body.

Visual Changes

N/A

Reviewer Notes

  • Kilo clients send only function tools, so their tools are unchanged. A free-model client that sends a client-run built-in tool type, such as Anthropic bash_* or Responses apply_patch, now has that tool removed. I chose this over a list of paid aliases, which would break as OpenRouter adds aliases.
  • Free models that read PDFs natively now get cloudflare-ai markdown instead of the raw PDF.
  • "Free" means isFreeModelExcludingCustomLlms(requestedModel), the same value that sets providerContext.isFree for OpenRouter and Vercel requests.

@chrarnoldus chrarnoldus self-assigned this Oct 22, 2026
@kilo-code-bot

kilo-code-bot Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

The incremental commit 953454a12 only extracts removePaidServerFeatures into its own module, moves its tests, and types OpenRouter plugins; the transform logic and its single call site under isFreeModelExcludingCustomLlms are unchanged, and I found no new defect.

Assumptions: [INFERENCE] The extraction is behavior-preserving: the moved function bodies (FREE_PLUGIN_IDS, getServerToolType, plugin/tool filtering, tool_choice handling, and the warning log payload) are identical to the previously reviewed inline versions, and removePaidServerFeatures has no importers other than apply-provider-specific-logic.ts:64 (verified by repository-wide grep), so dropping its export from that module is not a breaking API change. The new plugins?: OpenRouterPlugin[] field is only read/written inside remove-paid-server-features.ts (no other .plugins consumers in packages/web-shared/src/lib/ai-gateway), so the narrowed unknown type does not break other call sites. No tests, lint, or typecheck were executed because this review is read-only; the module-level FREE_PDF_PARSER_PLUGIN object is shared by reference across requests, but nothing mutates plugin objects, so no cross-request contamination was found.

Files Reviewed (5 files)
  • packages/web-shared/src/lib/ai-gateway/providers/remove-paid-server-features.ts
  • packages/web-shared/src/lib/ai-gateway/providers/remove-paid-server-features.test.ts
  • packages/web-shared/src/lib/ai-gateway/providers/apply-provider-specific-logic.ts
  • packages/web-shared/src/lib/ai-gateway/providers/apply-provider-specific-logic.test.ts
  • packages/web-shared/src/lib/ai-gateway/providers/openrouter/types.ts
Previous Review Summaries (3 snapshots, latest commit 5fc7dd9)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 5fc7dd9)

Status: No Issues Found | Recommendation: Merge

Executive Summary

The incremental change (5fc7dd93) removes the now-redundant isAnonymousUserId guard and its test, leaving paid-feature stripping gated only on isFreeModelExcludingCustomLlms; I verified in the only caller path that anonymous callers cannot reach this transform with a non-free model, so no defect is introduced.

Assumptions: [INFERENCE] (1) isAnonymousUserId(userId) was removed only as a redundant condition: applyProviderSpecificLogic is reachable solely through sendUpstreamAttempt from handleLlmProxyRequest (packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.ts:1016), and an anonymous context is created there only when isFreeModel(effectiveModelIdLowerCased) is true (llm-proxy.ts:646-651). isFreeModel differs from isFreeModelExcludingCustomLlms only for custom LLMs, which getProvider refuses for anonymous contexts (providers/get-provider.ts:134-140, surfaced as model-not-found at llm-proxy.ts:839-842) and which the auto-decision routing path never lets through (authenticatedRoutingCaller, llm-proxy.ts:408-413, 452-467), so the invariant "anonymous implies free model" holds at the transform. (2) Behavior for signed-in users is unchanged because the removed branch only matched userId values beginning with anon:. (3) The deleted anonymous test did not cover a distinct code path; the remaining free-model tests exercise the same transform, so the coverage loss is not a defect. No behavioral change was verified by execution; no commands were run.

Files Reviewed (2 files)
  • packages/web-shared/src/lib/ai-gateway/providers/apply-provider-specific-logic.ts
  • packages/web-shared/src/lib/ai-gateway/providers/apply-provider-specific-logic.test.ts

packages/web-shared/src/lib/ai-gateway/providers/openrouter/types.ts was unchanged since the previously reviewed commit and was not re-reviewed.

Previous review (commit b53e447)

Status: No Issues Found | Recommendation: Merge

Executive Summary

The incremental change (b53e44770) replaces blanket plugin deletion with a default-deny allowlist (response-healing, context-compression) and forces a free cloudflare-ai file-parser engine on OpenRouter free requests, resolving the previous SUGGESTION without introducing a new defect in the reviewed paths.

Assumptions: [INFERENCE] (1) cloudflare-ai is a valid, unbilled OpenRouter file-parser engine and the two allowlisted plugin ids are unbilled — these are external OpenRouter facts, not verifiable in this repo, and there was no manual test against OpenRouter per the PR description; because the plugin is now added to every free OpenRouter request (including requests with no files, and for messages/responses kinds), a wrong engine id or an endpoint that rejects plugins would affect all free OpenRouter traffic. (2) Free models that accept PDFs natively will now receive cloudflare-ai markdown instead of the raw PDF; the author states this tradeoff in the PR description, and it is a deliberate consequence of forcing an engine rather than leaving OpenRouter's billed default. (3) Plugin objects are shared by reference into the request body; no code in the repo mutates body.plugins after this transform, so the module-level FREE_PDF_PARSER_PLUGIN constant cannot be poisoned across requests in the reviewed call path. The previous inline SUGGESTION is fixed at this HEAD and is intentionally not carried forward.

Files Reviewed (2 files)
  • packages/web-shared/src/lib/ai-gateway/providers/apply-provider-specific-logic.ts
  • packages/web-shared/src/lib/ai-gateway/providers/apply-provider-specific-logic.test.ts

Previous review (commit 8b50747)

Status: 1 Issue Found | Recommendation: Address before merge

Executive Summary

The free/anonymous request hardening in applyProviderSpecificLogic correctly strips paid OpenRouter plugins and server tools before any other transform; the only finding is a non-blocking suggestion that blanket plugin removal also drops free plugins.

Assumptions: [INFERENCE] OpenRouter's response-healing/auto-router plugins are not separately billed; this is based on OpenRouter's documented plugin pricing, not on repository code. No bypass was found through :online model suffixes, fallback models, tool_choice handling, or auto-routed models, because the free check runs on the resolved model id and anonymous callers are already restricted to free models.

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 1
Issue Details (click to expand)

SUGGESTION

File Line Issue
packages/web-shared/src/lib/ai-gateway/providers/apply-provider-specific-logic.ts 287 Deleting all plugins also removes non-billed plugins (e.g. free response-healing), regressing JSON repair for free/anonymous requests; consider an allowlist of known-free plugin ids.
Files Reviewed (3 files)
  • packages/web-shared/src/lib/ai-gateway/providers/apply-provider-specific-logic.ts - 1 issue
  • packages/web-shared/src/lib/ai-gateway/providers/apply-provider-specific-logic.test.ts - 0 issues
  • packages/web-shared/src/lib/ai-gateway/providers/openrouter/types.ts - 0 issues

Fix these issues in Kilo Cloud


Reviewed by deepseek-flash-latest · Input: 53.2K · Output: 9.2K · Cached: 571.9K

Review guidance: REVIEW.md from base branch main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants