Skip to content

[ISSUE #7871] Add provider-native AI protocol routing - #7985

Merged
hengyuss merged 1 commit into
apache:masterfrom
hengyuss:refactor/ai-protocol-route-resolver
Oct 23, 2026
Merged

hengyuss merged 1 commit into
apache:masterfrom
hengyuss:refactor/ai-protocol-route-resolver

Conversation

@hengyuss

@hengyuss hengyuss commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Part of #7871.

Summary

  • introduce request-scoped protocol routes that explicitly retain client and target protocols
  • prefer Provider-native protocol passthrough and fall back to registered bidirectional converters
  • keep identity conversion in the route resolver instead of the converter registry
  • recreate protocol routing and conversion state when switching to a fallback Provider
  • wire the route resolver through Spring configuration

Tests

  • AiProtocolConverterFactoryTest
  • AiProtocolRouteResolverTest
  • AiProxyEngineTest
  • AiProxyEngineOrchestrationTest
  • AiProxyPluginConfigurationTest
  • AI Proxy unit regression: 274 tests passed
  • real HTTP compatibility regression: 11 tests passed

Make sure that:

  • You have read the contribution guidelines.
  • You submit test cases (unit or integration tests) that back your changes.
  • Your local test passed ./mvnw clean install -Dmaven.javadoc.skip=true.

@Aias00 Aias00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: #7985 [ISSUE #7871] Add provider-native AI protocol routing

Verdict: approved. This is the natural follow-up to #7910: it takes the identity case out of the converter registry and makes the client→provider route an explicit, request-scoped value. The tests are the strongest part.

What the change actually does

  • IdentityAiProtocolConverter is deleted. Identity is no longer something the registry can answer: AiProtocolConverterFactory.getConverter and supports no longer special-case source == target, and findConverter returns Optional.empty() instead of throwing.
  • AiProtocolRouteResolver owns the decision: native passthrough first (nativeProtocols.contains(clientProtocol) → AiProtocolConversion.identity), otherwise a registered directional converter, otherwise ShenyuException.
  • AiProtocolRoute is a record that validates itself in the compact constructor — a converter that claims targetProtocol() == "native" but returns a conversion whose request protocol is something else is rejected rather than silently sent upstream.
  • AiProxyEngine.prepare now holds a AiProtocolRoute and passes route.clientProtocol() into PreparedExchange; the failover path at copy.setProtocol(primary.clientProtocol()) is unaffected.

Verified

  • No dangling callers of the removed identity path: getConverter is referenced only inside the factory and its test after this change (search/code over apache/shenyu shows only AiProtocolConverterFactory and AiProtocolConverterFactoryTest; the getConverter hits in shenyu-plugin-jwt are unrelated classes). supports has no production caller at all — only provider.supports(...) at AiProxyEngine:241 — so dropping identity from it is behaviourally free.
  • AiProtocolRouteResolverTest covers the five cases that matter: native route wins without invoking the converter (assertFalse(invoked.get())), converted route binds the converted request and the reverse response adapter, registration order decides between two viable targets, no route → ShenyuException, and a lying converter → IllegalArgumentException from the record.
  • AiProtocolConverterFactoryTest is rewritten in step rather than deleted: testDoesNotTreatIdentityAsRegisteredConversion now asserts both supports(a, a) == false and that getConverter(a, a) throws, which is the contract the resolver depends on.
  • The starter still wires everything: aiProtocolRouteResolver is a @Bean @ConditionalOnMissingBean, aiProxyEngine takes it, and AiProxyPluginConfigurationTest asserts hasSingleBean(AiProtocolRouteResolver.class). Note this bean sits in the same plain @Configuration I flagged for #7917 — the ordering caveat applies, but it is pre-existing here and not worth widening this PR.

Non-blocking

  1. PreparedExchange.targetProtocol is written but never read. AiProxyEngine uses clientProtocol() at line 117 for failover but nothing consumes targetProtocol() (grep over the head file: lines 252, 273, 274 only). Either drop the component or use it in the log/metric line — right now it is a field that will silently drift.
  2. AiProtocolRoute can throw NullPointerException instead of IllegalArgumentException. The compact constructor does conversion.request().getProtocol(); if a converter returns a conversion with a null request, the failure surfaces as an NPE rather than the validation error this record exists to produce. Objects.requireNonNull(conversion.request(), …) first would keep the contract uniform.
  3. AiProtocolConverterFactory gained a stray blank line before the closing brace (the + at the end of the hunk) — harmless, but the diff has no reason to touch that line.
  4. The unsupported-route message still names protocols rather than the provider ("Unsupported AI protocol route from X to provider protocols [openai-chat]"). Same note I left on #7910; still not blocking, and now the message is built one layer closer to the provider so it would be cheap to include provider.name().

@hengyuss
hengyuss merged commit 31cf4fa into apache:master Oct 23, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants