Skip to content

AI gateway: rewrite the response model to the public ID for Kilo stealth models #8860

Closed

Description

@chrarnoldus

Problem

For Kilo-exclusive (stealth) models, the gateway sends the upstream request with the model's internal_id, and forwards the upstream response without touching its model field. The response therefore names the upstream model, not the public Kilo model ID the client requested:

  • chat completions: model on the body and on every chunk
  • messages: message.model on the body and on message_start
  • responses: response.model on the body and on the response.* events

The gateway already hides the upstream model name for stealth models elsewhere:

  • shouldRedactModelNameInMicrodollarUsage for usage rows
  • shouldRedactErrorResponse for error responses (packages/web-shared/src/lib/ai-gateway/kilo-exclusive-models.ts)

Successful responses are not covered.

This now matters for clients. Kilo CLI v2 takes the per-step routed model from the response model (Kilo-Org/kilocode#15366, packages/kilo-gateway-core/src/stream.ts). It stores the routed model, shows it, and groups usage stats by it. For a stealth model, that means the CLI displays and stores the upstream model ID.

Proposal

In rewriteModelResponse, when the requested model is a Kilo stealth model, rewrite model in the body and in the stream frames listed above to the requested public ID.

  • Keep kilo-auto/* as it is: there the routed model is intentionally visible.
  • Consider custom LLMs too (provider === 'custom', which shouldRedactModelNameInMicrodollarUsage also redacts). They are less sensitive, because the organization configured the upstream itself.

Acceptance criteria

  • For stealth models, no response frame on any shape (chat, messages, responses; stream and non-stream) contains the internal_id.
  • kilo-auto/* responses still report the routed model.
  • Tests cover all three shapes.

Context: AISAGE-789 (review of the Kilo CLI v2 gateway changes: Anaconda-Sandbox/gateway-resources#3).

Activity

  1. self-assigned this
    on Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions