You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(gateway): retry on another site when the dial fails #419
grid_site_route picks one site per request and never picks again. When the dial to that site fails, the client gets the upstream error, even if another candidate site could serve the model. Praxis retries endpoints within a cluster, but no retry crosses to another site's cluster. A 5xx response only evicts the prefix-affinity entry for that cluster (route.rs, settle).
The window this leaves: a site that becomes unreachable stays selectable until its signals age past load_window_ms or the praxis health check marks its cluster down. Requests picked for it in that window fail.
Proposal: on a connect failure to the picked site, before any request bytes are sent, pick once more from the remaining candidates, excluding the failed site. Pinned requests (stored Responses API state) are out of scope, since only the holding site can serve them.
Open questions:
whether the praxis filter API lets a route filter re-select the cluster after a connect failure, or whether this needs a praxis change;
the retry budget, and whether non-idempotent requests that fail before any byte is sent are safe to retry.
Activity