[gopaljigaur]

What changed

Rate limits. Run events now carry the provider's HTTP response headers and the rate limits read from them.

• StepFinished gets headers (the answering completion's) and rate_limit.
• A new CompletionRetry event (model, delay, status_code, headers, rate_limit, error) is emitted before each retry of a step's completion from feat(core): retry transient provider errors and time out stalled streams #95, streamed or not, and before a stalled stream's one retry. feat(core): retry transient provider errors and time out stalled streams #95 emitted nothing on a retry, so there was no event to extend. The compaction summary's retries emit none, since _ContextWindow has no run to emit to.
• ModelFallback (feat: fall back to other models on overload, outage or refusal #126) gets status_code, headers and rate_limit: the error's for a failure, the refused response's headers for a refusal.
• RateLimit (exported from cyclops) has remaining_requests, reset_requests, remaining_tokens, reset_tokens. RateLimit.from_headers(headers) reads OpenAI's x-ratelimit-* and Anthropic's anthropic-ratelimit-* headers and returns None when none are there. A header it can't read leaves its field None, so an odd value from an OpenAI-compatible server never fails a run.
• _Watched in fallback.py now flags only text and reasoning deltas as emitted, so a retry event in a stream doesn't stop the fallback after it.

Context usage. Agent.context_usage(session) -> Optional[ContextUsage] reports tokens (system prompt, the tools a run starts with, transcript), max_input_tokens and compaction_threshold (None without compaction). It builds the same _ContextWindow a run builds, through _context_window, so the semantics match: an unknown model gives None, or with compaction set raises the same ValueError with the register_model hint. It uses only _model_info and token_counter, so it makes no network request (#130). acompact and context_usage share a new Agent._request that notes the system prompt and tools on the window, and _ContextWindow.threshold() replaces the two inline threshold computations.

Docs (docs and website): a Rate limits section in Streaming, the event table, retries in Agents, a Context usage section under Context window, and the API reference.

What LiteLLM 1.105.1 exposes

Checked by answering LiteLLM's real requests from a patched httpx.AsyncClient.send:

Provider	Response: _hidden_params["additional_headers"]	Error
OpenAI	every header as llm_provider-<name>, plus unprefixed x-ratelimit-{limit,remaining}-{requests,tokens} (no reset)	RateLimitError, status_code=429, raw headers in litellm_response_headers
Anthropic	every header as llm_provider-<name>, plus the same four unprefixed headers mapped from anthropic-ratelimit-* (no reset)	the same
Gemini	empty: LiteLLM passes on no headers	the same; Gemini sends no rate limit headers
Bedrock	llm_provider-x-amzn-requestid and the like	the same; Bedrock sends no rate limit headers

Streams keep the same additional_headers on the response stream_chunk_builder assembles, directly and through a Router. Exceptions have no response_headers attribute in 1.105.1; the headers are in litellm_response_headers, which #95 already reads for retry-after.

Decisions

• Raw headers, as the SDKs give them. The OpenAI and Anthropic SDKs return raw httpx.Headers through with_raw_response, and the Vercel AI SDK puts a raw Record<string, string> on response.headers of each step. headers follows that: the provider's own names, with LiteLLM's llm_provider- prefix taken off. LiteLLM's unprefixed copies are left out since they duplicate the raw ones. Errors give the same names from litellm_response_headers.
• RateLimit field names follow OpenAI's headers (remaining_requests, reset_tokens), which LiteLLM also normalizes to. Resets are seconds from the response, as retry-after and the IETF RateLimit header draft give them: OpenAI's Go durations (6m0s, 120ms) are summed, and Anthropic's RFC 3339 times are measured from now, at least 0. LiteLLM normalizes no reset header itself.
• No limit fields. The issue asks for what is left and when it resets; headers has the limits for anyone who wants them.
• A new event for retries. No existing event fires on a retry. A retry callback is common (LangChain's on_retry, tenacity's before_sleep), and a stream consumer needs it to show a wait.
• The run's final error is unchanged. RunResult.error is LiteLLM's exception, which already has status_code and litellm_response_headers; the docs show RateLimit.from_headers(error.litellm_response_headers).
• Counting once per call. _ContextWindow counts each message once per window, and a window lives for one run (or one compact() call) because a truncate or compaction between runs invalidates its per-message counts. context_usage builds a window the same way acompact does, so each call counts the transcript once, as a run's first step does. Keeping a window on the session across runs would need invalidation for truncate() and other runs' compactions.
• compaction_threshold is ceil(threshold * max_input_tokens), the first token count at which compact_if_needed compacts.

Tests

tests/test_rate_limits.py, with OpenAI's and Anthropic's real responses and errors parsed by LiteLLM (no network):

• StepFinished.headers and rate_limit for both providers, JSON and streamed;
• a 429 with rate limit headers and retry-after emits CompletionRetry with the status, headers and limits, then the run answers, JSON and streamed;
• ModelFallback carries the 429's status and headers;
• a streamed retry still falls back (fails without the _Watched change);
• a response without rate limit headers has rate_limit=None;
• RateLimit.from_headers: Go durations, a past Anthropic reset, unreadable or missing headers.

tests/test_compaction.py: context_usage with compaction matches the tokens_before a manual compaction counts, with threshold 320 of 400, and makes no completion; without compaction it counts the same tokens with no threshold; an unknown model gives None, or ValueError with compaction set.

tests/test_model_info.py: context_usage for unmapped Ollama, Lemonade and Hugging Face models, directly and through a Router, sends no HTTP request, and a registered model gets its window.

uv run pytest -n 4: 751 passed, 1 xfailed. pre-commit run --all-files and mypy cyclops pass. The touched test files also pass on Python 3.10 and at the lowest direct dependencies (litellm 1.102.0).

Closes #75

[gopaljigaur]

No findings.