You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Bug: max_completion_tokens in llm_pool.defaults ignored when judge_panel is not configured #348
resolve_llm_config() — which merges defaults → model params — is never called
Ragas falls back to InstructorModelArgs.max_tokens = 1024 Works when: model-level parameters set + judge_panel configured (triggers resolve_llm_config())
Reproduce
Set llm_pool.defaults.parameters.max_completion_tokens: 500, no judge_panel
Run eval → effective limit is 1024, not 500
Fix: When llm_pool is present but no judge_panel, resolve primary model via resolve_llm_config() instead of falling back to system_config.llm.
Activity