Skip to content

fix(ai-gateway): fail custom LLM requests cleanly when credentials cannot be resolved - #7779

Closed
chrarnoldus wants to merge 1 commit into
mainfrom
kilo/custom-llm-credential-failures
Closed

chrarnoldus wants to merge 1 commit into
mainfrom
kilo/custom-llm-credential-failures

Conversation

@chrarnoldus

Copy link
Copy Markdown
Contributor

Summary

Two failures in custom LLM credential resolution threw out of getProvider, and handleLlmProxyRequest turned them into a 500:

  • Credentials that can't be decrypted (rotated BYOK_ENCRYPTION_KEY or corrupted ciphertext). These are now treated like the existing invalid-credentials case: logged, with the custom LLM skipped.
  • A failed Google service account token exchange (usually transient). This now returns a new custom-llm-temporarily-unavailable result, which the proxy answers with a 503. The user learns to retry instead of getting an internal error.

This is a prep PR for #7291.

Verification

  • No manual testing; covered by unit tests for both failure paths and the 503 mapping.

Visual Changes

N/A

Reviewer Notes

The error log includes only the public id and the error message, never credentials.

@chrarnoldus chrarnoldus self-assigned this Oct 16, 2026
@kilo-code-bot

kilo-code-bot Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

The custom LLM credential-failure handling is correct and consistently wired: decryption/schema failures fall through like the existing invalid-credentials case, service-account token-exchange failures produce a new custom-llm-temporarily-unavailable result mapped to a 503, and the new union member is only consumed by llm-proxy (the sole getProvider caller), so no exhaustive consumer breaks.

Files Reviewed (4 files)
  • packages/web-shared/src/lib/ai-gateway/providers/get-provider.ts
  • packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.ts
  • packages/web-shared/src/lib/ai-gateway/providers/get-provider.custom-llm.test.ts
  • packages/web-shared/src/lib/ai-gateway/handlers/llm-proxy.test.ts

Notes

  • Verified temporarilyUnavailableResponse() returns 503 and getGoogleServiceAccountAccessToken is the only importer of the mocked module, so the new test-level jest.mock factory does not break other exports.
  • Verified CustomLlmCredentialsSchema has exactly the four discriminants handled by the resolveCustomLlmAuth switch, and CustomLlm2/EncryptedData/authTag shapes used by the new code and tests exist.
  • The new return value is { kind: 'custom-llm-temporarily-unavailable' } and the llm-proxy guard is placed before const effectiveProviderContext = providerResult, preserving correct type narrowing.
  • No new memory-leak paths in changed code (the pre-existing unbounded authClients cache in google-service-account.ts is unchanged).

Reviewed by deepseek-v4.1-flash · Input: 59.5K · Output: 16.8K · Cached: 842.1K

Review guidance: REVIEW.md from base branch main

@chrarnoldus

Copy link
Copy Markdown
Contributor Author

We don't want this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant