Skip to content

[AISAGE-706] fix(ai-gateway): replace OpenRouter reasoning object for direct BYOK providers - #7754

Merged
chrarnoldus merged 4 commits into
mainfrom
fix/ai-gateway-direct-byok-reasoning-object
Oct 16, 2026
Merged

chrarnoldus merged 4 commits into
mainfrom
fix/ai-gateway-direct-byok-reasoning-object

Conversation

@chrarnoldus

@chrarnoldus chrarnoldus commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Since #7586, direct BYOK providers use the OpenRouter client by default. As a result, clients send OpenRouter's top-level reasoning object ({ effort, enabled, ... }) instead of reasoning_effort. Strict OpenAI-compatible upstreams reject reasoning as an unknown parameter. #7751 fixes this for OpenCode Go, which returned [unknown_parameter] invalid request body: json: unknown field "reasoning".

This PR applies the same fix to the other direct BYOK providers that still forward the object. A new shared helper, replaceReasoningObjectWithReasoningEffort in openrouter/request-helpers.ts, copies reasoning.effort into reasoning_effort, unless the request already sets reasoning_effort, and then deletes reasoning. It only touches Chat Completions. A Responses API reasoning object is valid and left alone.

Provider Before After
morph-byok, xiaomi-token-plan-ams, xiaomi-token-plan-sgp, alibaba-token-plan, neuralwatt Empty transformRequest: reasoning sent as is, effort never reached reasoning_effort Helper applied
chutes-byok, inceptron-byok, ollama-cloud, synthetic Copied effort to reasoning_effort but still sent reasoning Helper replaces the inline copy
zai-coding, byteplus-coding, kimi-coding Set thinking from reasoning, then still sent reasoning thinking is still derived from reasoning first, then the helper runs. In Kimi it runs before the existing reasoning_effort: 'none' removal, so a copied none is still removed
nvidia-byok, edenai Already removed reasoning; nvidia lost reasoning: {enabled: false} because it only copied effort Helper with { disableWithNoneEffort: true }, which maps enabled: false to reasoning_effort: 'none'. Only these two opt in: both were verified live to honor none
opencode-go Inline copy from #7751 Uses the helper

Unchanged: orcarouter keeps reasoning, because it is a router and likely understands OpenRouter's parameter.

The none mapping is opt-in because upstreams differ: nvidia and edenai honor reasoning_effort: 'none', but Chutes rejects it with a 400 literal_error.

Copying the effort into reasoning_effort matches what clients sent before #7586, when these providers used the openai-compatible client.

Upstream error logging

The Axiom investigation for this PR was slowed down by the upstream error log: direct BYOK failures appear as direct-byok returned error <status> with only the upstream model ID (for example glm-5.3-flash). The provider had to be guessed from the error format, and the rejected field from the response text. captureProxyError now also logs:

  • gatewayModel: the Kilo model ID after auto-routing (for example opencode-go/glm-5.3-flash), only when it differs from model
  • byokProviderIds: the user BYOK provider IDs serving the request (for example opencode-go)
  • requestFields / messageFields: top-level and per-message field names (never values) of the body as sent upstream, after provider transforms

The message prefix is unchanged, so existing startswith 'direct-byok returned error' queries keep working. Example from a local run of this branch (OpenCode Go omen-alpha, which still rejects reasoning until #7751 merges):

direct-byok returned error 400 {
  model: 'omen-alpha',
  gatewayModel: 'opencode-go/omen-alpha',
  byokProviderIds: 'opencode-go',
  requestFields: 'max_tokens,messages,model,reasoning',
  messageFields: 'content,reasoning,role',
  first4kOfResponse: '{"error":{"param":"reasoning",...,"message":"Upstream request failed: [unknown_parameter] invalid request body: json: unknown field \\"reasoning\\""}}'
}

Evidence

Verification

  • pnpm --filter @kilocode/web-shared typecheck: passes
  • Jest: providers/direct-byok/*, handlers/* and llm-proxy-helpers: 19 suites, 396 tests pass. The new direct-byok/reasoning-object.test.ts covers the helper, each provider listed above, thinking derivation and Kimi's none handling. New captureProxyError tests check the logged fields and that no field values (prompt or reasoning text) appear in the log.
  • I triggered a real OpenCode Go rejection through a local gateway on this branch and checked the log line shown above
  • oxlint on the changed files: clean; oxfmt and git diff --check: clean

@chrarnoldus chrarnoldus self-assigned this Oct 16, 2026
@kilo-code-bot

kilo-code-bot Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Re-reviewed the incremental commit 70b37810d (opt-in disableWithNoneEffort on replaceReasoningObjectWithReasoningEffort, wired into edenai/nvidia-byok, opencode-go on the default); the earlier disable-signal suggestion is resolved and no new issues were found on the changed lines.

The helper's default path is behavior-preserving (reasoning?.effort ?? undefined), so the providers left on the default (chutes-byok, inceptron-byok, ollama-cloud, synthetic, alibaba-token-plan, morph, neurowatt, xiaomi-token-plan-ams/sgp, opencode-go, byteplus-coding, kimi-coding, zai-coding) keep exactly their prior behavior, and the edenai refactor is semantically identical to the inline code it replaced. The prior suggestion to map reasoning: { enabled: false } to reasoning_effort: 'none' was addressed by an opt-in applied only to the two upstreams verified to accept none (nvidia, edenai), with Chutes documented as rejecting it, so it is treated as resolved rather than an outstanding finding.

Files Reviewed (5 files)
  • packages/web-shared/src/lib/ai-gateway/providers/openrouter/request-helpers.ts
  • packages/web-shared/src/lib/ai-gateway/providers/direct-byok/edenai.ts
  • packages/web-shared/src/lib/ai-gateway/providers/direct-byok/nvidia-byok.ts
  • packages/web-shared/src/lib/ai-gateway/providers/direct-byok/opencode-go.ts
  • packages/web-shared/src/lib/ai-gateway/providers/direct-byok/reasoning-object.test.ts
Previous Review Summary (commit 5d1a596)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 5d1a596)

Status: 1 Issue Found | Recommendation: Address before merge

Executive Summary

The reasoning-object replacement is implemented correctly, ordered correctly against isReasoningExplicitlyDisabled/thinking derivation, and well covered by tests; the single finding is a non-blocking suggestion that the shared helper drops the reasoning.enabled === false disable signal without the reasoning_effort: 'none' fallback that edenai.ts uses.

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 1
Issue Details (click to expand)

SUGGESTION

File Line Issue
packages/web-shared/src/lib/ai-gateway/providers/openrouter/request-helpers.ts 371 replaceReasoningObjectWithReasoningEffort deletes reasoning but only carries effort, so reasoning: { enabled: false } with no effort sends no disable signal for providers without a thinking/enable_thinking fallback (unlike edenai.ts:16-18)
Files Reviewed (17 files)
  • packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.ts
  • packages/web-shared/src/lib/ai-gateway/llm-proxy-helpers.ts
  • packages/web-shared/src/lib/ai-gateway/llm-proxy-helpers.test.ts
  • packages/web-shared/src/lib/ai-gateway/providers/openrouter/request-helpers.ts - 1 issue
  • packages/web-shared/src/lib/ai-gateway/providers/direct-byok/alibaba-token-plan.ts
  • packages/web-shared/src/lib/ai-gateway/providers/direct-byok/byteplus-coding.ts
  • packages/web-shared/src/lib/ai-gateway/providers/direct-byok/chutes-byok.ts
  • packages/web-shared/src/lib/ai-gateway/providers/direct-byok/inceptron-byok.ts
  • packages/web-shared/src/lib/ai-gateway/providers/direct-byok/kimi-coding.ts
  • packages/web-shared/src/lib/ai-gateway/providers/direct-byok/morph.ts
  • packages/web-shared/src/lib/ai-gateway/providers/direct-byok/neurowatt.ts
  • packages/web-shared/src/lib/ai-gateway/providers/direct-byok/ollama-cloud.ts
  • packages/web-shared/src/lib/ai-gateway/providers/direct-byok/synthetic.ts
  • packages/web-shared/src/lib/ai-gateway/providers/direct-byok/xiaomi-token-plan-ams.ts
  • packages/web-shared/src/lib/ai-gateway/providers/direct-byok/xiaomi-token-plan-sgp.ts
  • packages/web-shared/src/lib/ai-gateway/providers/direct-byok/zai-coding.ts
  • packages/web-shared/src/lib/ai-gateway/providers/direct-byok/reasoning-object.test.ts

Fix these issues in Kilo Cloud


Reviewed by deepseek-v4.1-flash · Input: 72.2K · Output: 19K · Cached: 887.7K

Review guidance: REVIEW.md from base branch main

@chrarnoldus

chrarnoldus commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor Author

E2E verification with live provider keys

I tested four providers with live keys: chutes-byok, inceptron-byok, nvidia-byok and edenai.

Setup: I ran ai-gateway locally with a seeded user whose BYOK keys were stored for those four providers. I sent requests through POST /api/v1/chat/completions in two ways:

  • Raw requests: no reasoning parameters; reasoning: {effort: "high"}; reasoning: {effort: "low"}; reasoning: {enabled: false}; and a history whose assistant message carries the legacy reasoning string plus reasoning_details.
  • SDK runs: @openrouter/ai-sdk-provider 3.1.0 with ai 7, a two-step tool-calling conversation with reasoning: {effort: "high"}, run through generateText and streamText.

For the baseline, I restored the provider files and request-helpers.ts to current main (which includes #7751) and restarted the gateway. All 28 cases (7 per provider) returned 200 before and after. The one change you can see is the one this PR intends:

Model reasoning: {enabled: false} on main On this branch
nvidia nemotron-3-super-120b-a12b 340 reasoning chars, 155 reasoning tokens: disable lost 0 chars, 0 tokens: disable honored
edenai openai/gpt-5.6-luna 0 reasoning tokens 0 reasoning tokens (same behavior, now through the helper)
chutes Kimi-K2.6-TEE 200, still reasons 200, still reasons. none is not sent, because Chutes rejects it with a 400 literal_error (checked directly against Chutes)
inceptron Kimi-K2.6 200 200

Upstream behavior, checked directly against each provider rather than through the gateway:

  • nvidia: honors both reasoning: {enabled: false} and reasoning_effort: "none".
  • edenai: honors reasoning_effort: "none" (0 reasoning tokens) and ignores the OpenRouter reasoning object.
  • chutes: ignores the object, scales reasoning with reasoning_effort (low 577 chars, high 886), and rejects none.
  • inceptron: ignores both.

None of these four rejects the reasoning object outright. For them this PR mainly carries the effort over and adds the nvidia disable fix; the strict-validator protection is aimed at upstreams like OpenCode Go.

Separate finding, not changed here: inceptron returns reasoning in the legacy reasoning string field rather than reasoning_content. The reasoning-content response transform therefore doesn't turn it into reasoning_details. The OpenRouter SDK still shows it, because it falls back to message.reasoning. On the way back, #7751 folds the legacy string into reasoning_content.

@chrarnoldus
chrarnoldus added this pull request to the merge queue Oct 16, 2026
Merged via the queue into main with commit d931ab7 Oct 16, 2026
29 checks passed
@chrarnoldus
chrarnoldus deleted the fix/ai-gateway-direct-byok-reasoning-object branch October 16, 2026 12:58
@chrarnoldus chrarnoldus changed the title fix(ai-gateway): replace OpenRouter reasoning object for direct BYOK providers [AISAGE-706] fix(ai-gateway): replace OpenRouter reasoning object for direct BYOK providers Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants