Repository navigation
Conversation
clear_user_turn() reset the live transcript but left the scheduled _end_of_turn_task running. When the endpointing delay elapsed, the bounce read the now-empty transcript and still committed the turn, so on_user_turn_completed ran with an empty message and AgentActivity created an empty user message plus an agent reply. Track the trigger of the pending decision and cancel it from _clear_user_turn() unless it is a manual commit (an explicit application decision that must keep its semantics). Also guard the bounce itself against committing an empty non-manual turn as a belt-and-braces check. Add a regression test driving VAD/STT events through a live session.
wait_for_idle() awaits the pending _end_of_turn_task behind an asyncio.shield. When clear_user_turn() cancels that task, the shield forwards CancelledError to the waiter, aborting wait_for_idle() and losing work queued behind it, such as a deferred tool reply. Swallow CancelledError only when the shielded decision itself was cancelled (a dropped/superseded turn); cancellation of the waiter still propagates because the shielded task keeps running. Add an end-to-end regression test with an idle waiter racing clear_user_turn().
| try: | ||
| await asyncio.shield(eou_task) | ||
| except asyncio.CancelledError: | ||
| # a dropped decision (clear_user_turn, or a newer trigger that | ||
| # superseded it) is idle, not a failed wait. cancellation of | ||
| # *this* waiter still propagates: the shielded task keeps running. | ||
| if not eou_task.cancelled(): | ||
| raise |
There was a problem hiding this comment.
🟡 Agent-only idle wait skips replacement decision
When _run_eou_detection supersedes a pending decision, wait_for_idle(wait_for_user=False) can return while its replacement remains active. The agent-only loop exits after the cancelled task if no speech is queued, letting the turn-limit callback run early.
Learn more
An end-of-turn decision is an asynchronous task that can be replaced by a later transcript or VAD event. AudioRecognition._run_eou_detection cancels the old task and installs a new one. An agent-only idle wait awaits the old task, catches its cancellation, then checks only speech; with no queued speech, wait_for_idle exits even though the replacement decision can still produce a response. The turn-limit path uses exactly this agent-only wait in _user_turn_exceeded_task.
Example: A final transcript schedules decision A. A newer final transcript replaces it with decision B while the turn-limit callback waits on A. With no speech queued yet, the waiter returns before B's delay ends and fires the callback prematurely.
Recommended fix: After swallowing the cancellation, recheck the current recognition task and keep the agent-only wait active until any replacement decision completes. Preserve the distinction between cancellation of the decision and cancellation of the waiter.
Was this helpful? React with 👍 or 👎 to provide feedback.
wait_for_idle() treated any cancellation of the shielded end-of-turn task as idle. When a newer trigger cancels decision A and installs B, an agent-only waiter (wait_for_user=False, used by the turn-limit path) returned while B was still pending, so the turn-limit callback could fire before the replacement decision committed. Loop in _wait_for_eou instead: after a cancelled decision, await the currently pending task if one was installed as a replacement. A dropped decision from clear_user_turn() clears the reference, so that path still exits as idle. Add a regression test with A superseded by B while an agent-only idle waiter is blocked on A.
fix(voice): drop cleared end-of-turn decisions without breaking idle waits
What breaks
AgentSession.clear_user_turn()(push-to-talk cancel / application-driven discard) resets the live transcript and turn state but leaves a scheduled_end_of_turn_taskrunning. When its endpointing delay elapses,_bounce_eou_taskreads the now-emptyself._audio_transcriptat commit time and still callson_end_of_turn.AgentActivitythen runson_user_turn_completedwith an empty message, appends an empty user message to history, and generates an agent reply for the turn the application just discarded. Reproduced deterministically: chat history[None, None, '', 'reply after cancel'], LLM inputs['hello world', ''], two speech handles.Review then surfaced two idle-wait regressions in the first fix:
wait_for_idle()awaits the pending decision behindasyncio.shield; cancelling it forwardedCancelledErrorto the waiter, aborting the idle wait and losing deferred tool replies (_ToolExecutor._deliver_reply)._run_eou_detectionsupersedes decision A with B, an agent-only waiter (wait_for_idle(wait_for_user=False), used by the turn-limit path) returned while B was still pending, so the turn-limit callback could fire early.Root cause
AudioRecognition._clear_user_turn()cancels the STT/commit path but not the already-scheduled end-of-turn decision,_bounce_eou_tasktrusts the transcript captured at scheduling time instead of the live transcript at commit time, andAgentActivity._wait_for_eou()cannot distinguish cancellation of the shielded decision from cancellation of the waiter, nor a dropped decision from one superseded by a replacement.Fix
vad,stt,manual) of the pending decision in_end_of_turn_triggerwhen_run_eou_detectionschedules_end_of_turn_task._clear_user_turn(), cancel and drop the pending task (and its trigger) unless it is a manual decision. Manualcommit_user_turn()is an explicit application decision and keeps its current semantics, including cancellation by resumed speech._bounce_eou_task: for non-manual triggers with STT enabled, never commit when the transcript is empty at commit time, mirroring the existing_run_eou_detectionentry guard.AgentActivity._wait_for_eou(), swallowCancelledErroronly when the shielded decision itself was cancelled (waiter cancellation still propagates), then loop: if a replacement decision is pending, await it. A cleared turn drops the reference, so the loop exits and reads idle.Tests
tests/test_agent_session.py:test_clear_user_turn_cancels_pending_end_of_turn— drives VAD start-of-speech, a final STT transcript, and VAD end-of-speech through a live session, then clears while the decision is pending; asserts noon_user_turn_completedinput, no empty LLM input, no scheduled reply speech, no history messages, and the pending task cancelled.test_clear_user_turn_does_not_cancel_wait_for_idle— startswait_for_idle()while the decision is pending, clears, and requires the waiter to complete normally while still producing no empty message/reply.tests/test_wait_for_idle_cancellation.py:test_superseded_end_of_turn_task_waits_for_replacement— blocks an agent-only idle waiter on decision A, replaces A with B and cancels A, asserts the waiter is still pending, then completes B and releases it.On base
e57a07b7: all three fail (assert [''] == [];Failed: clear_user_turn surfaced CancelledError through wait_for_idle();assert not idle_waiter.done()), 3 failed. With the fix: 3 passed;tests/test_agent_session.py112 passed;tests/test_wait_for_idle_cancellation.py4 passed; the broad relevant suite 415 passed; ruff clean.Note:
tests/test_eou_wait_span.pyhas pre-existing real-time flakiness in span-ordering assertions, reproduced with this change stashed; unrelated.