You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
AI gateway: rewrite the response model to the public ID for Kilo stealth models #8860
For Kilo-exclusive (stealth) models, the gateway sends the upstream request with the model's internal_id, and forwards the upstream response without touching its model field. The response therefore names the upstream model, not the public Kilo model ID the client requested:
chat completions: model on the body and on every chunk
messages: message.model on the body and on message_start
responses: response.model on the body and on the response.* events
The gateway already hides the upstream model name for stealth models elsewhere:
shouldRedactModelNameInMicrodollarUsage for usage rows
shouldRedactErrorResponse for error responses (packages/web-shared/src/lib/ai-gateway/kilo-exclusive-models.ts)
Successful responses are not covered.
This now matters for clients. Kilo CLI v2 takes the per-step routed model from the response model (Kilo-Org/kilocode#15366, packages/kilo-gateway-core/src/stream.ts). It stores the routed model, shows it, and groups usage stats by it. For a stealth model, that means the CLI displays and stores the upstream model ID.
Proposal
In rewriteModelResponse, when the requested model is a Kilo stealth model, rewrite model in the body and in the stream frames listed above to the requested public ID.
Keep kilo-auto/* as it is: there the routed model is intentionally visible.
Consider custom LLMs too (provider === 'custom', which shouldRedactModelNameInMicrodollarUsage also redacts). They are less sensitive, because the organization configured the upstream itself.
Acceptance criteria
For stealth models, no response frame on any shape (chat, messages, responses; stream and non-stream) contains the internal_id.
kilo-auto/* responses still report the routed model.
Tests cover all three shapes.
Context: AISAGE-789 (review of the Kilo CLI v2 gateway changes: Anaconda-Sandbox/gateway-resources#3).
Activity