You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[AISAGE-757] fix(ai-gateway): bill message_start input tokens on early stream end - #8551
Messages API streams only recorded usage from message_delta. A stream that ended before message_delta arrived (client disconnect, upstream disconnect, or the gateway duration budget) was recorded with 0 tokens. That includes a client that stops reading at content_block_stop, after it already has the full response. OpenRouter and Vercel recover cost through the generation lookup; providers without one, such as Martian for the stealth models, were billed $0.
message_start already reports the input and cache token counts. The parser now records them and merges them with message_delta usage, preferring the message_delta value when it is present. A stream that stops early is billed for at least its prompt. Output tokens still come from message_delta only, so an early stop does not bill the output generated so far.
The "No usage in Messages API stream" warning still fires when message_delta usage is missing.
Verification
Local stack with the real Martian upstream: streamed stealth/claude-sonnet-4.6 on /messages and aborted the client at content_block_stop. Before: 0 tokens, $0. After: input tokens recorded and a nonzero cost.
Visual Changes
N/A
Reviewer Notes
message_start input tokens are 0 on Vercel, so the merge keeps the message_delta value there.
The change keeps message_start usage as a fallback when a Messages API stream ends before message_delta, and the three new tests cover early end, omitted delta fields, and delta precedence. I verified the parser and both approval fixtures: the vercel-messages.log.resp.sse fixture has all cache fields in message_delta, so its snapshot is unchanged.
Assumptions:[INFERENCE] Two unproven concerns, neither reproduced: (1) if a provider sent message_delta.usage.input_tokens that already includes cached tokens but omitted the cache fields, the merge would add message_start cache counts on top and over-count input (the intended shape is delta input null, as tested); (2) startUsage retains message_start's output_tokens (typically 1, asserted in the new test), so early-ended streams bill a token or two of output even though the PR description says output comes from message_delta only. Also noted: with the condition now !deltaUsage, the "SUSPICIOUS: No usage in Messages API stream" warning fires for exactly the early-ended streams this PR records usage for, which is stated as intentional but will add Sentry noise on client aborts. No memory-leak risk in the change (one extra per-stream object reference).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Messages API streams only recorded usage from
message_delta. A stream that ended beforemessage_deltaarrived (client disconnect, upstream disconnect, or the gateway duration budget) was recorded with 0 tokens. That includes a client that stops reading atcontent_block_stop, after it already has the full response. OpenRouter and Vercel recover cost through the generation lookup; providers without one, such as Martian for the stealth models, were billed $0.message_startalready reports the input and cache token counts. The parser now records them and merges them withmessage_deltausage, preferring themessage_deltavalue when it is present. A stream that stops early is billed for at least its prompt. Output tokens still come frommessage_deltaonly, so an early stop does not bill the output generated so far.The "No usage in Messages API stream" warning still fires when
message_deltausage is missing.Verification
stealth/claude-sonnet-4.6on/messagesand aborted the client atcontent_block_stop. Before: 0 tokens, $0. After: input tokens recorded and a nonzero cost.Visual Changes
N/A
Reviewer Notes
message_startinput tokens are 0 on Vercel, so the merge keeps themessage_deltavalue there.