Skip to content

fix(voice): cancel the pending end-of-turn decision when clearing a user turn - #7940

Open
BlueX888 wants to merge 3 commits into
livekit:mainfrom
BlueX888:fix/clear-user-turn-cancels-eou
Open

BlueX888 wants to merge 3 commits into
livekit:mainfrom
BlueX888:fix/clear-user-turn-cancels-eou

Conversation

@BlueX888

@BlueX888 BlueX888 commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

fix(voice): drop cleared end-of-turn decisions without breaking idle waits

What breaks

AgentSession.clear_user_turn() (push-to-talk cancel / application-driven discard) resets the live transcript and turn state but leaves a scheduled _end_of_turn_task running. When its endpointing delay elapses, _bounce_eou_task reads the now-empty self._audio_transcript at commit time and still calls on_end_of_turn. AgentActivity then runs on_user_turn_completed with an empty message, appends an empty user message to history, and generates an agent reply for the turn the application just discarded. Reproduced deterministically: chat history [None, None, '', 'reply after cancel'], LLM inputs ['hello world', ''], two speech handles.

Review then surfaced two idle-wait regressions in the first fix:

  1. wait_for_idle() awaits the pending decision behind asyncio.shield; cancelling it forwarded CancelledError to the waiter, aborting the idle wait and losing deferred tool replies (_ToolExecutor._deliver_reply).
  2. Treating every cancellation as idle was too broad: when _run_eou_detection supersedes decision A with B, an agent-only waiter (wait_for_idle(wait_for_user=False), used by the turn-limit path) returned while B was still pending, so the turn-limit callback could fire early.

Root cause

AudioRecognition._clear_user_turn() cancels the STT/commit path but not the already-scheduled end-of-turn decision, _bounce_eou_task trusts the transcript captured at scheduling time instead of the live transcript at commit time, and AgentActivity._wait_for_eou() cannot distinguish cancellation of the shielded decision from cancellation of the waiter, nor a dropped decision from one superseded by a replacement.

Fix

  • Track the trigger (vad, stt, manual) of the pending decision in _end_of_turn_trigger when _run_eou_detection schedules _end_of_turn_task.
  • In _clear_user_turn(), cancel and drop the pending task (and its trigger) unless it is a manual decision. Manual commit_user_turn() is an explicit application decision and keeps its current semantics, including cancellation by resumed speech.
  • Belt-and-braces in _bounce_eou_task: for non-manual triggers with STT enabled, never commit when the transcript is empty at commit time, mirroring the existing _run_eou_detection entry guard.
  • In AgentActivity._wait_for_eou(), swallow CancelledError only when the shielded decision itself was cancelled (waiter cancellation still propagates), then loop: if a replacement decision is pending, await it. A cleared turn drops the reference, so the loop exits and reads idle.

Tests

tests/test_agent_session.py:

  • test_clear_user_turn_cancels_pending_end_of_turn — drives VAD start-of-speech, a final STT transcript, and VAD end-of-speech through a live session, then clears while the decision is pending; asserts no on_user_turn_completed input, no empty LLM input, no scheduled reply speech, no history messages, and the pending task cancelled.
  • test_clear_user_turn_does_not_cancel_wait_for_idle — starts wait_for_idle() while the decision is pending, clears, and requires the waiter to complete normally while still producing no empty message/reply.

tests/test_wait_for_idle_cancellation.py:

  • test_superseded_end_of_turn_task_waits_for_replacement — blocks an agent-only idle waiter on decision A, replaces A with B and cancels A, asserts the waiter is still pending, then completes B and releases it.

On base e57a07b7: all three fail (assert [''] == []; Failed: clear_user_turn surfaced CancelledError through wait_for_idle(); assert not idle_waiter.done()), 3 failed. With the fix: 3 passed; tests/test_agent_session.py 112 passed; tests/test_wait_for_idle_cancellation.py 4 passed; the broad relevant suite 415 passed; ruff clean.

Note: tests/test_eou_wait_span.py has pre-existing real-time flakiness in span-ordering assertions, reproduced with this change stashed; unrelated.

clear_user_turn() reset the live transcript but left the scheduled
_end_of_turn_task running. When the endpointing delay elapsed, the bounce
read the now-empty transcript and still committed the turn, so
on_user_turn_completed ran with an empty message and AgentActivity created
an empty user message plus an agent reply.

Track the trigger of the pending decision and cancel it from
_clear_user_turn() unless it is a manual commit (an explicit application
decision that must keep its semantics). Also guard the bounce itself
against committing an empty non-manual turn as a belt-and-braces check.
Add a regression test driving VAD/STT events through a live session.
@BlueX888
BlueX888 requested a review from a team as a code owner October 19, 2026 05:35
devin-ai-integration[bot]

This comment was marked as resolved.

wait_for_idle() awaits the pending _end_of_turn_task behind an
asyncio.shield. When clear_user_turn() cancels that task, the shield
forwards CancelledError to the waiter, aborting wait_for_idle() and
losing work queued behind it, such as a deferred tool reply.

Swallow CancelledError only when the shielded decision itself was
cancelled (a dropped/superseded turn); cancellation of the waiter still
propagates because the shielded task keeps running. Add an end-to-end
regression test with an idle waiter racing clear_user_turn().

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Devin Review

Comment on lines +2119 to +2126
try:
await asyncio.shield(eou_task)
except asyncio.CancelledError:
# a dropped decision (clear_user_turn, or a newer trigger that
# superseded it) is idle, not a failed wait. cancellation of
# *this* waiter still propagates: the shielded task keeps running.
if not eou_task.cancelled():
raise

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Agent-only idle wait skips replacement decision

When _run_eou_detection supersedes a pending decision, wait_for_idle(wait_for_user=False) can return while its replacement remains active. The agent-only loop exits after the cancelled task if no speech is queued, letting the turn-limit callback run early.

Learn more

An end-of-turn decision is an asynchronous task that can be replaced by a later transcript or VAD event. AudioRecognition._run_eou_detection cancels the old task and installs a new one. An agent-only idle wait awaits the old task, catches its cancellation, then checks only speech; with no queued speech, wait_for_idle exits even though the replacement decision can still produce a response. The turn-limit path uses exactly this agent-only wait in _user_turn_exceeded_task.

Example: A final transcript schedules decision A. A newer final transcript replaces it with decision B while the turn-limit callback waits on A. With no speech queued yet, the waiter returns before B's delay ends and fires the callback prematurely.

Recommended fix: After swallowing the cancellation, recheck the current recognition task and keep the agent-only wait active until any replacement decision completes. Preserve the distinction between cancellation of the decision and cancellation of the waiter.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

wait_for_idle() treated any cancellation of the shielded end-of-turn
task as idle. When a newer trigger cancels decision A and installs B,
an agent-only waiter (wait_for_user=False, used by the turn-limit path)
returned while B was still pending, so the turn-limit callback could
fire before the replacement decision committed.

Loop in _wait_for_eou instead: after a cancelled decision, await the
currently pending task if one was installed as a replacement. A dropped
decision from clear_user_turn() clears the reference, so that path still
exits as idle. Add a regression test with A superseded by B while an
agent-only idle waiter is blocked on A.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant