You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Feat: Service Principal (client credentials) auth mode to the ADLS Gen2 sink connector #2270
The kafka-connect-azure-datalake sink currently supports three auth modes (config/AuthModeSettings.scala): Credentials (account key), ConnectionString, and Default (DefaultAzureCredential, worker-level).
There is no way to authenticate to storage with a per-connector Service Principal (Entra ID App Registration: client_id / client_secret / tenant_id) using client-credentials OAuth.
I'd like to add a ServicePrincipal auth mode backed by ClientSecretCredentialBuilder from com.azure.identity,
which is already a project dependency (azure-identity, see project/Dependencies.scala) and already used for DefaultAzureCredentialBuilder in auth/DatalakeClientCreator.scala. No new dependency is required.
Rational
Usually security standards require secrets to be short-lived and scoped. A storage account key or connection string is a long-lived,
account-wide credential. DefaultAzureCredential is a worker-level setting, not something that can be configured per connector.
A Service Principal with a scoped RBAC role and short-lived OAuth tokens, configurable per connector instance, closes
that gap while keeping all the existing sink functionality.
Proposed change
I think the actual changes should be quite small and dependeny-free.
Configuration — config/AuthModeSettings.scala: add an AuthMode.ServicePrincipal(clientId, tenantId, clientSecret) case, plus config keys (e.g. connect.datalake.azure.client.id, .tenant.id, .client.secret),
wired into getAuthMode.
Authentication — auth/DatalakeClientCreator.scala: add a createDataLakeClientWithServicePrincipal(...) path using new ClientSecretCredentialBuilder().clientId(..).tenantId(..).clientSecret(..).build().
Tests & docs — unit tests for the new auth mode, and an update to the public connector docs (auth mode
table currently lists Credentials / ConnectionString / Default).
I'm happy to implement this myself and submit a PR per the project's "Contributing" guidance.
Before starting, I wanted to check:
is this something the maintainers would be open to accepting,
and is there any existing work or design preference I should be aware of (e.g. naming of config keys, whether ClientSecretCredential should be exposed directly or wrapped in a broader "explicit credential" auth mode)?
Activity
marcel-radusch-vf commentedon Oct 6, 2026
Hi,
since I didn't get any response yet. I try to tag some of the previous contributors for guidance.
Casey W Crites (@caseycrites) Caio Oliveira (@caiooliveiraeti) Hubert Behaghel (@behaghel)
Are you willing to accept or review this contribution or are you not open for it?