Skip to content

fix(plugin): evict cached fault-tolerance handles when explicitly cleared (#7914) - #7925

Merged
Aias00 merged 2 commits into
apache:masterfrom
konlue:fix/fault-tolerance-blank-handle-clear
Oct 22, 2026
Merged

Aias00 merged 2 commits into
apache:masterfrom
konlue:fix/fault-tolerance-blank-handle-clear

Conversation

@konlue

@konlue konlue commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Fixes #7914.

Summary

Admin accepts a rule update whose handle is an explicit empty string, and Gson parses that input to null. The fault-tolerance handlers then diverge: Resilience4J removes the registry but leaves the previous cached handle in place (the next request NPEs in Resilience4JBuilder re-parsing the blank raw handle), RateLimiter keeps the previous handle enforcing in CACHED_HANDLE, and Sentinel NPEs in handlerRule before updating anything.

Modifications

  • Follow the idiom already merged for the Request/Rewrite/ModifyResponse/ParamMapping handlers in [ISSUE #7759] Evict cached rule handles when explicitly cleared. #7794: a blank (empty or whitespace) handle explicitly clears the rule, while a null handle (omitted field of a selective update) keeps its existing semantics.
    • Resilience4JHandler / RateLimiterPluginDataHandler: evict the cached handle; both plugins already pass through when the cached handle is absent, so the stale enforcement and the Resilience4JBuilder NPE are both gone.
    • SentinelRuleHandle: delegate to removeRule, which evicts the cached handle and strips the rule's flow/degrade entries from the Sentinel rule managers; the parse / checkData / reload path only runs for non-blank handles.
  • No runtime code changes: with the cache evicted, every cleared rule degrades to the same behavior as a rule without a handle.

Verifying this change

  • Each handler test: valid handle cached → handlerRule("") evicts it (fails on the previous code with the issue's stale handle / NullPointerException), whitespace also clears, and a subsequent valid update re-caches; Sentinel additionally asserts the flow/degrade rules for the key are stripped.

  • Regression green: Resilience4JPluginTest 3/3, RateLimiterPluginTest 4/4, SentinelPluginTest 5/5.

  • Checkstyle passes:

    ./mvnw -pl <three modules> checkstyle:check

    clean.

@Aias00 Aias00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix itself is correct and the three handlers are now consistent in intent. There is one blocker, though: the build is red because of a Checkstyle violation in the new test, so this cannot be merged as is.

Blocking — CI is red on this PR (not on master)

Both failures come from the same violation introduced by this PR:

pr_build -> FAILURE
  [ERROR] Resilience4JHandlerTest.java:65:9: Distance between variable 'cache' declaration
          and its first usage is 5, but allowed 3. Consider making that variable final ...
  [ERROR] Failed to execute goal maven-checkstyle-plugin:3.4.0:check (validate)
          on project shenyu-plugin-resilience4j: You have 1 Checkstyle violation.
build (shenyu-integrated-test-k8s-gateway-api-http) -> FAILURE
  (same violation; the reactor stops at shenyu-plugin-resilience4j)

testHandlerRuleExplicitClearEvictsPreviousRule declares the supplier at the top of the method and only touches it five statements later:

Supplier<CommonHandleCache<String, Resilience4JHandle>> cache = Resilience4JHandler.CACHED_HANDLE;
ruleData.setSelectorId("1");
ruleData.setId("clear-test");
ruleData.setHandle("{\"urlPath\":\"test\"}");
resilience4JHandler.handlerRule(ruleData);
Assertions.assertNotNull(cache.get().obtainHandle("1_clear-test"));

Either mark it final (final Supplier<CommonHandleCache<String, Resilience4JHandle>> cache = ... — the rule is configured with ignoreFinal, which is why the message suggests it) or move the declaration down to just above its first use. The latter reads better here since the three setXxx calls do not depend on it.

What I verified as correct

  • Eviction is the right lever, because all three plugins fail open when the handle is absent. RateLimiterPlugin:75, SentinelPlugin:60 and Resilience4JPlugin:67 all do if (Objects.isNull(handle)) { return chain.execute(exchange); }. So removing the cached handle stops enforcement without turning the request into a 500 — which is exactly the semantics "an explicitly cleared handle" should have. I checked all three rather than assuming, since a plugin that dereferenced the handle would have made this fix a 500-generator.
  • Sentinel delegating to removeRule(ruleData) is the strongest part of the change. removeRule already does cache eviction plus FlowRuleManager/DegradeRuleManager filtering inside synchronized (RULE_UPDATE_LOCK), so the clear path shares the same lock as the update path instead of being a second, unlocked mutation of the global Sentinel rule sets. Re-implementing the eviction inline would have raced against handlerRuleShouldNotLoseConcurrentUpdates.
  • Resilience4JHandler tears down the registry before the blank check, so a cleared handle drops the circuit breaker / bulkhead / rate limiter instances as well as the cached handle — matching what removeRule does.
  • The Sentinel NPE was real before this change: GsonUtils.fromJson("") returns null, so the old sentinelHandle.checkData() threw on the sync thread. SentinelPlugin:63 also calls checkData(), so a stale entry could have kept failing per request.
  • Global test state is handled: SentinelRuleHandleTest's @BeforeEach clears both Sentinel rule managers, so assertEquals(1, FlowRuleManager.getRules().size()) is not order-dependent.

Non-blocking

  1. null and "" now behave differently, in all three handlers. The blank branch lives inside Optional.ofNullable(ruleData.getHandle()).ifPresent(...), so "" and " " evict while a null handle silently does nothing. Whether a cleared handle reaches the plugin as "" or as null depends on the admin column ('' vs NULL) and on the serializer, which makes the fix's coverage depend on data shape rather than on intent. Since the issue is about a cleared handle, I would treat null as a clear too — a rule whose handle disappeared should not keep enforcing the previous one — or add a test that pins the current choice deliberately.
  2. removeRule still has the same defect the PR is fixing, for the other two handlers. RateLimiterPluginDataHandler.removeRule and Resilience4JHandler.removeRule both wrap removeHandle in ifPresent(...), so deleting a rule whose handle is already null leaves the cached handle enforcing forever. The new blank branch avoids this by calling CACHED_HANDLE.get().removeHandle(...) directly, which is right — but it means the two paths now disagree. Fixing removeRule the same way would make this class of bug disappear and would let handlerRule's blank branch simply delegate to it, like Sentinel does.
  3. Resilience4JHandler's new import, org.apache.commons.lang3.StringUtils, is inserted after org.apache.shenyu.common.enums.PluginEnum. The other two files put org.apache.commons first; Checkstyle does not enforce the order here, but the file now groups imports differently from its siblings.
  4. Resilience4JHandler.handlerRule calls Resilience4JRegistryFactory.remove(key) on every update, so each rule change resets that key's breaker/bulkhead/limiter state. That is pre-existing and out of scope, but the new clear branch makes it worth a one-line comment that the reset is intentional rather than an accident of the clear path.
  5. The RateLimiter and Resilience4J tests leave their clear-test key cached when they finish (the last assertion re-caches it). Harmless because the key is test-specific, but an @AfterEach removeHandle("1_clear-test") would keep the static caches as clean as the Sentinel test's rule managers.

@konlue

konlue commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

All addressed. The blocker is gone: the supplier is declared right above its first use, and checkstyle is green across the three modules.

On the non-blocking points, I went with your lean: null is now treated as a clear as well.

  • All three handlerRule implementations check isBlank(ruleData.getHandle()) up front (including null) and delegate to removeRule.
  • removeRule in Resilience4J / RateLimiter evicts unconditionally, so deleting a rule whose handle is already null can no longer leave the cached handle enforcing, and the two paths can no longer disagree. That also let Sentinel's clear path and the other two read the same.

Plus the smaller items:

  • The StringUtils import is grouped with its siblings.
  • The registry reset carries an "intentional" comment.
  • The tests clean up their keys in @AfterEach.
  • Each handler has a null-handle regression test.

@Aias00 Aias00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: #7925 fix(plugin): evict cached fault-tolerance handles on rule/selector delete (re-review)

Verdict: approved — Checkstyle violation resolved, null and blank handle evictions unified across all fault-tolerance plugins, and all CI suites are green.

What was verified in this update

  1. Checkstyle compliance:
    • Fixed variable distance violation in Resilience4JHandlerTest.testHandlerRuleExplicitClearEvictsPreviousRule by declaring final Supplier<...> cache directly adjacent to usage.
  2. Unified null/blank handle eviction semantics:
    • Across RateLimiterPluginDataHandler, Resilience4JHandler, and SentinelRuleHandle, StringUtils.isBlank(ruleData.getHandle()) now covers both empty strings and null values consistently, immediately delegating to removeRule(ruleData).
    • removeRule(...) unbinds removeHandle from Optional.ofNullable(ruleData.getHandle()), ensuring that deleting a rule with a null handle reliably cleans up cached handles without leaking state.
  3. Comprehensive test assertions & cleanup:
    • Added unit tests for null handle evictions:
      • handlerRuleNullHandleEvictsPreviousRule & removeRuleWithoutHandleStillEvicts in RateLimiter.
      • testHandlerRuleNullHandleEvictsPreviousRule & testRemoveRuleWithoutHandleStillEvicts in Resilience4J.
      • handlerRuleNullHandleEvictsCacheAndEnforcement in Sentinel.
    • Added @AfterEach teardown in test classes to keep static handle caches clean.
  4. CI Status:
    • All 18 checks succeeded (including pr_build, Checkstyle, rat_check, and CodeQL), 0 failures.

@Aias00
Aias00 merged commit 5901356 into apache:master Oct 22, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Clearing fault-tolerance rule handles leaves stale enforcement or crashes updates and requests

2 participants