Skip to content

Streamed Anthropic thinking is doubled in history, breaking its signature #108

Open

Description

@gopaljigaur

With LiteLLM 1.105.1, a streamed Anthropic step with thinking and a tool call puts the thinking text into the history twice. The next request sends the doubled text with the original signature, so it no longer matches it, and Anthropic checks the signature of the last assistant turn in a tool loop. Non-streamed runs are not affected.

Cause (LiteLLM): llms/anthropic/chat/handler.py emits the whole accumulated thinking text again on the signature_delta chunk, and stream_chunk_builder (get_combined_thinking_content) appends it to the parts it already has.

Repro: feed LiteLLM's Anthropic SSE parser thinking_delta "Add ", thinking_delta "them.", signature_delta "sig-1", then a tool_use block, and run agent.stream("1 + 2?") with an add tool over two steps. The second request carries:

{"type": "thinking", "thinking": "Add them.Add them.", "signature": "sig-1"}

where it should carry "Add them.". test_streamed_anthropic_tool_loop_sends_back_the_signed_thinking in tests/test_provider_options.py (strict xfail) runs exactly this through AsyncHTTPHandler.post.

The fix belongs in LiteLLM; cyclops should not work around it. Remove the xfail once a LiteLLM release fixes it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions