Skip to content

fix(vscode): render streamed text at frame cadence - #14002

Merged
marius-kilocode merged 2 commits into
mainfrom
investigate-text-streaming-latency-reduction
Sep 10, 2026
Merged

marius-kilocode merged 2 commits into
mainfrom
investigate-text-streaming-latency-reduction

Conversation

@marius-kilocode

Copy link
Copy Markdown
Collaborator

What Problem This Solves

Streamed assistant text in the VS Code sidebar stepped visibly instead of tracking the stream. The webview throttled text and reasoning parts to a fixed 100 ms (TEXT_RENDER_THROTTLE_MS), so live text could repaint at most ~10 times per second, while the host scheduler already delivered deltas on a 16 ms focused lane. The TUI renders at roughly frame cadence, so VS Code felt slower for the same model output.

Why This Change Was Made

createThrottledValue now accepts a reactive interval. Text and reasoning parts pass ~one frame (16 ms) while the part is streaming and fall back to the existing 100 ms once it settles. Completed history keeps the slow cadence, so scrolling old messages stays cheap. The default interval is unchanged, so other callers such as shell-rolling-results.tsx are unaffected.

This also stays safe with many concurrent agents. The host already paces each lane: focused 16 ms, visible 50 ms, background 150-400 ms with per-agent backoff. Background sessions stream slower than 100 ms, so their render cadence is unchanged. The webview now matches the focused lane instead of overriding it with 100 ms.

User Impact

Streamed text and reasoning appear smoothly as tokens arrive instead of in 100 ms steps. Total response time is unchanged, since it is model-bound. Background Agent Manager sessions are unaffected.

Evidence

Measured in an isolated VS Code instance with a real model and the compiled webview bundle verified to contain the change. Same prompt and viewport, 3 runs each, sampling rendered text per animation frame.

Inter-update render gap:

Metric Before After Change
Median 100 ms 41 ms -59%
p25 100 ms 25 ms -75%
Minimum 94 ms 15 ms -84%
Visible updates/sec 8.5 15.0 +77%

Before, every gap was >=86 ms (0% under 75 ms). After, about 40% of gaps are <=33 ms. The p90 gap stays near 100 ms because token arrival now sets the ceiling, not the render throttle.

No CPU regression in Chromium metric deltas over the stream: ScriptDuration 0.73-1.00 s before vs 0.87-0.93 s after; RecalcStyleDuration and LayoutDuration within run-to-run noise. Zero webview console errors; final answer rendered correctly with headings, lists, and code blocks.

Validation

  • bun test --conditions=browser ./src/components/ from packages/kilo-ui: 72 pass, 0 fail (includes 3 new cadence tests).
  • Manual: streamed a normal reply in the isolated VS Code sidebar, confirmed smooth text, correct markdown, and no console errors.

Limitations

  • p90 still tracks token arrival, so bursty providers can still gap.
  • Measured for one focused stream, not for many simultaneously visible Agent Manager sessions; those use the host's 50 ms visible lane.

The webview throttled streamed text and reasoning parts to a fixed
100ms, so live text stepped visibly instead of tracking the stream,
while the host scheduler already emitted deltas on a 16ms focused lane.

Throttle at ~one frame while a part is streaming and keep the 100ms
cadence once it settles, so completed history stays cheap. Parameterize
createThrottledValue with a reactive interval, defaulting to 100ms, so
other callers are unchanged.
@kilo-code-bot

kilo-code-bot Bot commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (2 files)
  • packages/kilo-ui/src/components/tool-utils.test.ts
  • packages/kilo-ui/src/components/tool-utils.ts
Previous Review Summary (commit 8d5cae6)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 8d5cae6)

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 1
Issue Details (click to expand)

SUGGESTION

File Line Issue
packages/kilo-ui/src/components/tool-utils.ts 26 Switching to the idle interval can delay the last streamed tokens
Files Reviewed (4 files)
  • .changeset/streaming-text-frame-cadence.md - 0 issues
  • packages/kilo-ui/src/components/message-part.tsx - 0 issues
  • packages/kilo-ui/src/components/tool-utils.test.ts - 0 issues
  • packages/kilo-ui/src/components/tool-utils.ts - 1 issue

Fix these issues in Kilo Cloud


Reviewed by grok-4.6 · Input: 111.4K · Output: 9.6K · Cached: 70.1K

Review guidance: REVIEW.md from base branch main

When a part settles, the throttle interval grows from 16ms to 100ms and
the effect recomputed the trailing timeout with the longer interval,
delaying the last streamed token by up to ~90ms.

Flush the pending value immediately when the interval increases, and add
a regression test that transitions one throttle instance from streaming
to settled.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants