Skip to content

[Bugfix][Router] Retry a failed dynamic config slowly instead of every check - #1184

Open
shubhamkapoor01 wants to merge 4 commits into
vllm-project:mainfrom
shubhamkapoor01:fix/dynamic-config-persistent-failure
Open

shubhamkapoor01 wants to merge 4 commits into
vllm-project:mainfrom
shubhamkapoor01:fix/dynamic-config-persistent-failure

Conversation

@shubhamkapoor01

Copy link
Copy Markdown

If a new config fails to apply and the file stays as is, the watcher reapplies it on every check (10s by default), rebuilding service discovery each time. A failed config is now skipped for the next 30 checks before it is retried. It usually fails the same way again, but not always: a callbacks module can become importable later, so it isn't skipped forever. Any change to the config is applied at once, and going back to the last good config is still reapplied, as in #1177. Follows up #1177, from @chrikrah's review.

This is on top of #1177, so its two commits show here until that one is merged; only the last commit is new.


  • Make sure the code changes pass the pre-commit checks.
  • Sign-off your commit by using -s when doing git commit
  • Try to classify PRs for easy understanding of the type of changes, such as [Bugfix], [Feat], and [CI].

Signed-off-by: shubhamkapoor01 <66124414+shubhamkapoor01@users.noreply.github.com>
Signed-off-by: shubhamkapoor01 <66124414+shubhamkapoor01@users.noreply.github.com>
…y check

Signed-off-by: Shubham Kapoor <66124414+shubhamkapoor01@users.noreply.github.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a retry mechanism for failed configuration reloads in the DynamicConfigWatcher. When a configuration fails to apply (for example, if a callbacks module is not yet importable), the watcher will now skip a set number of checks before retrying, rather than failing indefinitely or spamming retries. Corresponding unit tests have been added to verify this behavior. The feedback points out that when reconfigure_all raises an exception, it is caught and re-raised to an outer block that logs a misleading "Error loading config file" message. It is recommended to handle this exception directly in the inner block to provide a more accurate "Error applying config" warning.

Comment thread src/vllm_router/dynamic_config.py Resolved
Signed-off-by: Shubham Kapoor <66124414+shubhamkapoor01@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant