Core, REST: Add BearerAuthManager by adutra · Pull Request #18754 · apache/iceberg

adutra · 2026-11-02T12:00:19Z

Add a dependency-free AuthManager that sets an Authorization: Bearer header on REST requests, as defined in RFC 6750.

The token is either read from a local file that is re-read periodically, or provided statically in the configuration. Obtaining and renewing the token is left to an external producer, such as a Kubernetes projected service account token or a sidecar.

The manager is selected with rest.auth.type=bearer. Tokens are validated against the RFC 6750 grammar, so that a malformed file cannot corrupt the Authorization header. If a reload fails, the last valid token is kept.

When rest.auth.type is not set, the legacy token property now infers the Bearer manager, instead of the OAuth2 manager (the credential property however still infers the OAuth2 manager).

Fixes #13550.

Add a dependency-free `AuthManager` that sets an `Authorization: Bearer` header on REST requests, as defined in RFC 6750. The token is either read from a local file that is re-read periodically, or provided statically in the configuration. Obtaining and renewing the token is left to an external producer, such as a Kubernetes projected service account token or a sidecar. The manager is selected with `rest.auth.type=bearer`. Tokens are validated against the RFC 6750 grammar, so that a malformed file cannot corrupt the Authorization header. If a reload fails, the last valid token is kept. When `rest.auth.type` is not set, the legacy `token` property now infers the Bearer manager, instead of the OAuth2 manager (the `credential` property however still infers the OAuth2 manager). Fixes apache#13550.

adutra · 2026-11-02T12:59:09Z

Design doc: https://docs.google.com/document/d/1SJpdaQP4cGKWpgUfCuRNeomA0c7c2tDyDtu8MjI5yXk/edit?tab=t.0

…tials grant Add `org.apache.iceberg.rest.auth.oauth2.OAuth2Manager`, an `AuthManager` that authenticates against any OAuth2 authorization server with the `client_credentials` grant and renews access tokens before they expire by running the grant again. It is selected explicitly by class name (`rest.auth.type=org.apache.iceberg.rest.auth.oauth2.OAuth2Manager`) and configured with `rest.auth.oauth2.*` properties; the legacy oauth2 manager is unchanged. Compared to the legacy manager, the new one discovers the token endpoint from the issuer URL (RFC 8414 or OpenID Connect Discovery), supports the `client_secret_basic` and `client_secret_post` authentication methods, and does not use token exchange for refresh. Static tokens, token exchange and tokens vended by the catalog server in table responses are not supported. Session contexts carrying OAuth2 properties get a self-contained session; nothing is inherited from the catalog configuration. The legacy properties `credential`, `oauth2-server-uri`, `scope`, `audience` and `resource` are translated with a warning, and the token endpoint falls back to the catalog's `v1/oauth/tokens` endpoint when neither a token endpoint nor an issuer URL is configured. Important: the `token` property is not supported anymore by this manager, but is supported by the `BearerAuthManager`, see apache#18754. The suggested migration path for people using `token` is therefore to use the `BearerAuthManager`. The implementation is built on the Nimbus OAuth SDK, and requests to the authorization server go through the catalog's `RESTClient`, which gains an `unwrap(Class)` method for that purpose. Runtime bundle `LICENSE` files and dependency baselines are updated accordingly.

adutra · 2026-11-02T18:30:53Z

FYI @rdblue @danielcweeks @nastra @sungwy @c-thiel @zeroshade @singhpk234

…tials grant Add `org.apache.iceberg.rest.auth.oauth2.OAuth2Manager`, an `AuthManager` that authenticates against any OAuth2 authorization server with the `client_credentials` grant and renews access tokens before they expire by running the grant again. It is selected explicitly by class name (`rest.auth.type=org.apache.iceberg.rest.auth.oauth2.OAuth2Manager`) and configured with `rest.auth.oauth2.*` properties; the legacy oauth2 manager is unchanged. Compared to the legacy manager, the new one discovers the token endpoint from the issuer URL (RFC 8414 or OpenID Connect Discovery), supports the `client_secret_basic` and `client_secret_post` authentication methods, and does not use token exchange for refresh. Static tokens, token exchange and tokens vended by the catalog server in table responses are not supported. Session contexts carrying OAuth2 properties get a self-contained session; nothing is inherited from the catalog configuration. The legacy properties `credential`, `oauth2-server-uri`, `scope`, `audience` and `resource` are translated with a warning, and the token endpoint falls back to the catalog's `v1/oauth/tokens` endpoint when neither a token endpoint nor an issuer URL is configured. Important: the `token` property is not supported anymore by this manager, but is supported by the `BearerAuthManager`, see apache#18754. The suggested migration path for people using `token` is therefore to use the `BearerAuthManager`. The implementation is built on the Nimbus OAuth SDK, and requests to the authorization server go through the catalog's `RESTClient`, which gains an `unwrap(Class)` method for that purpose. Runtime bundle `LICENSE` files and dependency baselines are updated accordingly.

danielcweeks · 2026-11-03T19:10:23Z

+The token file must exist and contain a valid token when the catalog is initialized. It must contain the token
+verbatim; leading and trailing whitespace is removed. Files larger than 256 KiB, and tokens containing characters
+not allowed by RFC 6750, are rejected. If a later reload fails, the last valid token keeps being used: deleting the
+file does not revoke the token. Symbolic links are followed. The file must be present at the same path on every node
+that accesses the catalog or its tables, e.g. on Spark executors when using the S3 remote signer.


Suggested change

The token file must exist and contain a valid token when the catalog is initialized. It must contain the token

verbatim; leading and trailing whitespace is removed. Files larger than 256 KiB, and tokens containing characters

not allowed by RFC 6750, are rejected. If a later reload fails, the last valid token keeps being used: deleting the

file does not revoke the token. Symbolic links are followed. The file must be present at the same path on every node

that accesses the catalog or its tables, e.g. on Spark executors when using the S3 remote signer.

The token file must exist and contain a valid token when the catalog is initialized; leading and trailing whitespace is removed. Files larger than 256 KiB, and tokens containing characters

not allowed by RFC 6750, are rejected. If a later reload fails, the last valid token keeps being used: deleting the

file does not revoke the token. Symbolic links are followed. The file must be present at the same path on every node

that accesses the catalog or its tables, e.g. on Spark executors when using the S3 remote signer.

Why are we defining a limit like 256KiB? Is that from the RFC or an arbitrary limit? Seems like we're over-specifying if this isn't grounded in a requirement.

danielcweeks · 2026-11-03T19:14:58Z

+        if (properties.containsKey(OAuth2Properties.CREDENTIAL)) {
+          warnInferredAuthType(AuthProperties.AUTH_TYPE_OAUTH2, OAuth2Properties.CREDENTIAL);
          authType = AuthProperties.AUTH_TYPE_OAUTH2;
+        } else if (properties.containsKey(OAuth2Properties.TOKEN)) {


Doesn't this change the default behavior for those who currently set token? If that's the case, we can't do this.

danielcweeks · 2026-11-03T19:27:13Z

+        .toString();
+  }
+
+  private static Path parseTokenFile(String tokenFile) {


I'm confused, what is this parsing?

danielcweeks · 2026-11-03T19:29:45Z

+  private final Cache<BearerAuthConfig, BearerAuthSession> sessionCache =
+      Caffeine.newBuilder().expireAfterAccess(Duration.ofHours(1)).executor(Runnable::run).build();
+
+  public BearerAuthManager(String ignored) {}


Why do we have this constructor?

danielcweeks · 2026-11-03T19:40:23Z

+ * current token of a {@link BearerTokenSource}. The token is fetched from the source on every
+ * request.
+ */
+record BearerAuthSession(BearerTokenSource source) implements AuthSession {


Why are we using record instead of class here? It feels like it's more for convenience than what it actually represents. I would go with class.

danielcweeks · 2026-11-03T19:48:30Z

+  /**
+   * Maximum token file size. Large enough for JWTs signed with the largest post-quantum signature
+   * schemes.
+   */
+  static final int MAX_FILE_SIZE = 256 * 1024;


This seems like unnecessary complexity. If they're large enough to not be able to be signed, then how would the end up in a valid representation.

danielcweeks · 2026-11-03T19:52:33Z

+   * @param reloading whether a thread has claimed the current reload
+   * @param failing whether the last reload attempt failed, used to log on transitions only
+   */
+  private record State(String token, long readAtNanos, boolean reloading, boolean failing) {}


This is a somewhat questionable use a record. Yes, the class represents state values, but it also has behavior and is mutating the state. Conceptually, this aligns more with a class than a record.

danielcweeks · 2026-11-03T19:53:26Z

+    this(path, reloadInterval, System::nanoTime);
+  }
+
+  FileTokenSource(Path path, Duration reloadInterval, LongSupplier nanoTime) {


Looking at the config and this class, it doesn't look like we're respecting the existing refresh enabled property.

danielcweeks · 2026-11-03T23:13:36Z

            CatalogProperties.URI,
            "ignored",
+            AuthProperties.AUTH_TYPE,
+            AuthProperties.AUTH_TYPE_OAUTH2,


This is a good indicator that we're breaking existing behavior. I think we'll probably need to have people opt into bearer behavior as opposed to the inference as it stands in the PR.

github-actions Bot added core docs labels Nov 2, 2026

This was referenced Nov 2, 2026

Support for Reading Bearer Token from File in Iceberg Catalog Configuration #13550

Closed

Core: Support file-based OAuth2 bearer tokens for the REST catalog (token-path): Closes #13550 #17389

Open

singhpk234 self-requested a review November 2, 2026 16:45

adutra mentioned this pull request Nov 2, 2026

Core, REST: Add standards-based OAuth2Manager using the client credentials grant #18756

Open

danielcweeks reviewed Nov 3, 2026

View reviewed changes

Navigation Menu

Repository navigation

Core, REST: Add BearerAuthManager - #18754

Core, REST: Add BearerAuthManager#18754
adutra wants to merge 1 commit into
apache:mainfrom
adutra:bearer-auth-manager

adutra commented Nov 2, 2026

Uh oh!

adutra commented Nov 2, 2026

Uh oh!

adutra commented Nov 2, 2026

Uh oh!

danielcweeks Nov 3, 2026

Uh oh!

danielcweeks Nov 3, 2026

Uh oh!

danielcweeks Nov 3, 2026

Uh oh!

danielcweeks Nov 3, 2026

Uh oh!

danielcweeks Nov 3, 2026

Uh oh!

danielcweeks Nov 3, 2026

Uh oh!

danielcweeks Nov 3, 2026

Uh oh!

danielcweeks Nov 3, 2026

Uh oh!

danielcweeks Nov 3, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

adutra commented Nov 2, 2026

Uh oh!

adutra commented Nov 2, 2026

Uh oh!

adutra commented Nov 2, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants