[gopaljigaur]

LiteLLM's model info lookups send a synchronous HTTP request for some providers, and Cyclops ran them on the event loop. This reads LiteLLM's local model map for those providers instead.

Where LiteLLM 1.105.1 goes to the network

_get_model_info_helper asks the provider's model-info config before the static map, and Hugging Face has its own branch. Probing every provider in litellm.provider_list with the HTTP clients patched to fail:

Provider	get_model_info, supports_*	get_supported_openai_params
ollama, ollama_chat	POST /api/show to OLLAMA_API_BASE or localhost, for a model the map lacks	local
lemonade	GET /models/{model}, always, even for the 5 models in the map	local
huggingface	GET huggingface.co/{model}/raw/main/config.json, always	local

No other provider reaches the network from these lookups. token_counter is local for all of them.

Cyclops call sites, before this change

• compaction._window, so _context_window (feat: retry a completion that overflows the context window #120) on every run, the compaction window (feat: summarize older messages near the model's input limit #118), the summarizing model's window and Agent.compact(): litellm.get_model_info, or Router.get_router_model_info per deployment. A Router registers its Ollama deployments as stubs in the map when it is built, so through a router only Lemonade and Hugging Face deployments reached the network.
• _native_schema (response_model with native tools): get_model_info.
• litellm.supports_response_schema (response_model without tools).
• Safe: _caches_at_markers (only asks for Anthropic and Bedrock), _takes_param and _lacks_tools (get_supported_openai_params), _takes_tool_message_images (get_llm_provider only).

What changed

• providers._SERVER_INFO_PROVIDERS lists those four providers, next to the module's other provider sets. _model_info(model, provider) reads litellm.model_cost under provider/model, else model, for them, using an entry only when its litellm_provider is that provider or unset, as LiteLLM's _check_provider_match does (so ollama/gpt-4o never reads OpenAI's entry). Every other provider calls litellm.get_model_info as before.
• _takes_input, the input check from feat: accept images, files and audio as run input #127, is now _model_info(...).get(flag) is not False. Other providers get LiteLLM's normalization, which fixes ids the direct map read missed, such as bedrock/converse/....
• _native_schema uses _model_info. A new _response_schema reads the map flag for those providers and calls litellm.supports_response_schema otherwise.
• _window uses _model_info. Through a router, a deployment of those providers gets its own model_info merged over the map entry, as get_router_model_info merges them. Other deployments still go through get_router_model_info.
• Docs: the Context window introduction of the agents guide (docs and website) says an unmapped local model gets no limits for compaction or overflow recovery. It shows register_model for ollama/qwen3:4b and says registering also stops LiteLLM's own per-completion /api/show requests. The multimodal guide describes the input check's lookup.

Decisions

• The map, not a thread. A thread would still send the request, to the wrong host when api_base points elsewhere, and would need the capability checks moved out of the sync _completion_kwargs. Reading the map is local and keeps everything sync.
• Only for the providers that go to the network. I compared a map-only read with LiteLLM's lookups over the 3,034 mapped chat model names. supports_response_schema differs for 334 of them (the providers it supports globally, a fallback to the bare model's entry, provider info), and the windows differ for 20. So every other provider keeps LiteLLM's own lookups unchanged. For all 29 mapped Ollama entries, under ollama/ and ollama_chat/, the map read gives the same windows and flags as get_model_info.
• Behaviour for a model with no local info is unchanged. With no server reachable, LiteLLM's answer had max_input_tokens=None, so compaction raised its ValueError naming litellm.register_model, and overflow recovery was skipped. Both still happen. What changes is a running Ollama server: its context_length is no longer picked up, so an unmapped Ollama model needs litellm.register_model for compaction. Mapped Lemonade models now get their window and supports_response_schema from the map, where LiteLLM ignored the map and asked the server.

Not covered (LiteLLM's own calls)

• Inside litellm.acompletion, LiteLLM prices each response, which sends /api/show for an unmapped Ollama model (4 requests per non-streamed completion in a probe). Registering the model avoids that. _usage reads the response_cost LiteLLM sets, so its completion_cost fallback isn't reached for LiteLLM's responses, streamed ones included.
• Building a litellm.Router asks each Ollama, Lemonade and Hugging Face deployment's server once.
• chatgpt/ and github_copilot/ run OAuth device-code requests in get_llm_provider. That is credential setup the completion needs anyway.

Tests

tests/test_model_info.py. httpx's Client.send and AsyncClient.send are patched to record each URL and fail. Each test uses a fresh unmapped model name for ollama, ollama_chat, lemonade and huggingface, alone and through a router:

• a run with response_model, reasoning_effort and prompt_caching, with and without tools, so every capability check runs: no request;
• an overflow: one completion, the run ends with ContextWindowExceededError, no request;
• compaction: the run and compact() raise the register_model ValueError before any completion, no request; the same for an unmapped summarizing model;
• the input check with an image part: no request;
• ollama/gpt-4o reads no entry, ollama/llama2 reads its own;
• a registered Ollama model gets its window and is compacted; a mapped Lemonade model gets its map window; a router deployment's model_info sets its window. None of them sends a request.

Without the fix, 29 of the 40 fail. The 11 that pass anyway are the Ollama router cases (covered by the Router's stubs) and the registered Ollama model.

tests/test_provider_options.py puts _response_schema into the memoized check table, and checks it decides as litellm.supports_response_schema for the mapped models there.

tests/test_multimodal.py adds a bedrock/converse/ id to the unsupported input cases.

uv run pytest (690 passed), uv run pre-commit run --all-files and uv run mypy cyclops pass. The touched test files also pass on Python 3.10.

Closes #130

[gopaljigaur]

Two findings, details inline:

1. _model_info should be the single local model info helper. #127's _takes_input should call it, and its bare-name fallback should check the entry's provider (ollama/gpt-4o now reads OpenAI's gpt-4o).
2. The docs note covers compaction but not overflow recovery, which is now silently off for unmapped Ollama models, including the docs' own ollama/qwen3:4b. A register_model example would help, along with the fact that registering also stops LiteLLM's per-completion /api/show requests.

Checked and fine:

• The provider probe is complete. I blocked every socket and DNS lookup and called get_model_info, five supports_* functions, get_supported_openai_params and get_llm_provider for all 165 providers, with mapped and unmapped names, with and without every *_API_BASE env var set. Only ollama and ollama_chat (unmapped models only), lemonade and huggingface reach the network, plus the chatgpt and github_copilot OAuth in get_llm_provider. Only OllamaModelInfo, Ollama's completion config and LemonadeChatConfig define get_model_info; Hugging Face has its own branch in _get_model_info_helper. The env vars change only the host.
• The Ollama change is acceptable. Before, the lookup went to OLLAMA_API_BASE or localhost, ignoring the deployment's api_base, and blocked the loop. Ollama's runtime info carries no supports_response_schema, so the structured output paths are unchanged.
• Router merge. It matches get_router_model_info for mapped ollama/llama3 with and without model_info.max_input_tokens, for base_model in model_info and in litellm_params, and for ollama_chat/llama3. No request is sent.
• Memoization. _native_schema and _response_schema go through #125's _holds with the same stale-after-register_model edge it documents. _window is not memoized, so a registered window is read at once.
• Tests. The 40 new tests pass. With main's cyclops/ they give 29 failed and 11 passed, exactly the Ollama router cases and the registered model, as the body says.
• Remaining LiteLLM-side requests. Router builds and the OAuth providers are fine as they are. The per-completion /api/show requests deserve an upstream LiteLLM report, since its pricing ignores api_base. On the cyclops side, the docs note in finding 2 is enough; no separate issue is needed.
• Checks. uv run pytest -p no:xdist: 637 passed, 1 xfailed. Pre-commit and mypy cyclops pass, and CI is green. The confidentiality grep and the dash check are clean. --is-ancestor origin/main is false: main gained #128 and #132 since. A local merge is clean and passes (639 passed), so it needs a rebase.

[gopaljigaur]

One docs nit, inline. Everything else is verified at 146d968.

• One helper. _takes_input now returns _model_info(...).get(flag) is not False. For the four network providers, the map read takes provider/model, then the bare name, and only accepts an entry whose litellm_provider is the model's provider or unset. That is the same rule as LiteLLM's _check_provider_match. ollama/gpt-4o now reads nothing, and ollama/llama2 still gets its window. test_another_providers_entry_is_not_read and the bedrock/converse/moonshotai.kimi-k2.5 audio case fail on 110885d and pass here.
• No network. I blocked every socket and DNS lookup and ran _model_info, _native_schema, _response_schema, _takes_input (vision, PDF, audio) and _window for all 165 providers, with mapped and unmapped names, with and without the *_API_BASE env vars. Only chatgpt and github_copilot reach the network, through the OAuth in get_llm_provider. Over every mapped chat key, _model_info reaches the network only for github_copilot keys, for the same reason. The other request in the process is LiteLLM's remote cost map fetch at import, which is unrelated to this PR.
• #127's decisions. azure/gpt-5.1-chat, ft:gpt-4o-2024-11-20, ft:gpt-4o-2024-08-06:my-org::abc123, databricks/databricks-claude-sonnet-4-5, deepinfra/meta-llama/Llama-3.2-11B-Vision-Instruct, ollama/llama3.2-vision, ollama/llava and gpt-3.5-turbo all still get images. o3-mini (vision) and bedrock/moonshotai.kimi-k2.5 (audio) are still refused. Over every mapped chat model and all three flags, the old map read and the new helper decide alike: 0 newly refused, 0 newly allowed. The only change is the converse route, now refused for audio as the map says.
• Docs. The context window introduction now covers compaction and overflow, with the ollama/qwen3:4b snippet keyed by the model string. The multimodal guides describe the shared lookup.
• Rebase. git range-diff shows 110885d equal to 8b6c671 apart from the providers.py context from #127, plus the new 146d968. --is-ancestor origin/main (8f77ba0) holds.
• Checks. uv run pytest -n 4: 690 passed, 1 xfailed. Pre-commit and mypy cyclops pass, and CI is green on 3.10 to 3.14. The confidentiality grep and the dash check are clean.

[gopaljigaur]

No findings.