[chrarnoldus]

Summary

Fixes non-streaming Gemini responses through the gemini-thought reasoning transform, which custom LLMs on Vertex AI's OpenAI-compatible endpoint use today and #8488's direct Vertex BYOK will use.

Problem. With thoughts enabled, a non-streaming response puts the thoughts and the answer in one message.content string and flags the whole message with extra_content.google.thought: true. Only the thoughts are wrapped in tags:

"content": "<think>\n**My Thought Process…**\n…\n</think>\n\n1081",
"extra_content": { "google": { "thought": true, "thought_signature": "…" } }

The transform follows the thought flag, so it moved the entire string, answer included, into reasoning_details, and the client received an empty answer. Streaming is unaffected: Vertex flags thought chunks separately there and sends no tags.

Fix.

• Request side: non-streaming requests now ask for the tag explicitly with google.thought_tag_marker: "think". Vertex's docs say no tags are returned unless this is set, although Vertex currently sends <think> anyway, so this pins the separator. Streaming requests don't get it. With a marker set, Vertex also tags streamed chunks and puts the closing tag in the first answer chunk.
• Response side: when content flagged as a thought starts with <think> and contains </think>, only the tagged part becomes a reasoning.text detail and the rest stays as content. Content flagged as a thought without tags keeps the old behavior. Thought signatures, on the message or on tool calls, are still mapped to encrypted details.

The underlying inconsistency is being reported to Google separately: the message is flagged as a thought even though its content also contains the answer.

Part of AISAGE-745.

Verification

• Against Vertex AI (global, 2026-10-26), sending requests built with applyReasoningDetailsTransform and passing responses through rewriteModelResponse_ChatCompletions:

  • gemini-3.1-pro-preview: content is "1081", with one 2,148-character reasoning.text detail and the encrypted signature.
  • gemini-3.5-flash with a tool: content is "" with the read tool call, plus a 720-character reasoning.text detail and the tool call's encrypted signature.

• Checked Vertex's behavior first:

  • Without streaming, thought_tag_marker is honored (a custom marker changes the tags), and the default tag is think.
  • With streaming and no marker, thought chunks are flagged and untagged.
  • With streaming and a marker, tags appear inside the chunks, which is why the request side only sets it without streaming.

• New tests:

  • splitting a tagged message with a message-level signature;
  • splitting a tagged message that precedes tool calls with a tool-call signature;
  • untagged content flagged as a thought;
  • the marker is requested only without streaming.

  rewriteModelResponse, apply-provider-specific-logic, custom LLM and build-direct-provider suites pass (286 tests). tsgo typecheck for web-shared and apps/web, and oxlint on the changed files, pass.

Visual Changes

N/A

Reviewer Notes

• Turning off thought summaries for non-streaming requests was the fallback. It isn't needed because the split works. Signatures arrive either way.
• Models used: Claude Opus 5.5 (kilo/anthropic/claude-opus-5.5), with no subagent model overrides.

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

The Gemini thought-tag change is small and self-contained: the request now opts into thought_tag_marker only for non-streaming requests, and the response transform splits tagged thoughts from the answer while preserving the old fallback for untagged content. No correctness, security, or performance defects were found in the changed code.

Assumptions: [INFERENCE] The split relies on Vertex returning thought content that begins exactly with <think> (no leading whitespace) and on the message carrying extra_content.google.thought: true; the PR description states this shape was verified against Vertex AI. [INFERENCE] The google.thought_tag_marker request field is accepted by the OpenAI-compatible endpoint per the PR's live verification; it cannot be confirmed from repository code. No tests were executed in this read-only review.

Files Reviewed (4 files)

• packages/web-shared/src/lib/ai-gateway/providers/apply-provider-specific-logic.test.ts
• packages/web-shared/src/lib/ai-gateway/providers/apply-provider-specific-logic.ts
• packages/web-shared/src/lib/ai-gateway/reasoning-details-transform.ts
• packages/web-shared/src/lib/ai-gateway/rewriteModelResponse.test.ts

---

_{Reviewed by deepseek-flash-latest · Input: 74.7K · Output: 25.8K · Cached: 1.3M}

_{Review guidance: REVIEW.md from base branch main}