acoustic-carpenter-78188
04/30/2026, 4:57 PMconnector.py.
This PR brings OAuth M2M and OIDC federation to the connector while preserving the existing per-namespace tenancy story, and refactors the existing PAT path so all four modes share one extension point.
What changes were proposed in this pull request?
A unified DatabricksAuth strategy abstraction in a new databricks_auth.py module, plus three new strategies and a refactor of the existing PAT path:
| Strategy | What it uses | Where credentials live |
| ------------------------------- | -------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
| PATAuth | Personal Access Token (existing flow) | databricks-token k8s secret in workflow namespace, with FLYTE_DATABRICKS_ACCESS_TOKEN fallback |
| OAuthM2MAuth | Service Principal client_id + client_secret | databricks-oauth k8s secret in workflow namespace, with env-var fallback |
| OIDCConnectorIRSAAuth (Model 1) | The connector pod's own projected JWT (e.g. EKS IRSA) | Federation policy in Databricks bound to the connector SA |
| OIDCNamespaceSAAuth (Model 2) | Per-workflow-namespace ServiceAccount JWT minted via Kubernetes TokenRequest API | Federation policy in Databricks bound to the workflow SA, discovered from SA labels and annotations |
Each strategy implements auth_type, get_bearer_token(session), invalidate_cache(), and describe(). select_auth(...) resolves task config -> connector env var -> auto-detect; build_auth(...) reconstructs the strategy from DatabricksJobMetadata so long-running jobs can refresh their token transparently on a 401 response.
Important note for reviewers: PAT is being refactored
This PR is more than a feature addition. It refactors the PAT support that landed via #3394 from a direct call into get_databricks_token(...) into a PATAuth strategy that lives alongside the new modes. The behaviour, env vars, custom-secret-name task field, and cross-namespace lookup semantics are preserved end to end (new tests cover all of those scenarios), but the code path now flows through select_auth -> PATAuth.get_bearer_token -> get_databricks_token. If you reviewed #3394 you may want to read connector.py and the new tests/test_databricks_token.py with this lens; the goal is one extension point for all four modes instead of two.
Files changed
• plugins/flytekit-spark/flytekitplugins/spark/databricks_auth.py (new): strategy abstraction, four strategies, select_auth, build_auth, token cache, OIDC discovery cache, validators.
• `plugins/flytekit-spark/flytekitplugins/spark/connector.py`: DatabricksJobMetadata carries auth context for refresh; create calls `select_auth`; get and delete go through a new _request_with_auth helper that refreshes on 401 and falls back to the stored token for older job metadata.
• `plugins/flytekit-spark/flytekitplugins/spark/task.py`: DatabricksV2 gains optional per-task overrides (databricks_auth_type, databricks_client_id, databricks_oauth_secret, databricks_oidc_token_file, databricks_oidc_audience). All optional, all backward compatible.
• plugins/flytekit-spark/tests/test_databricks_auth.py (new): 56 tests covering every strategy, the resolver, auto-detection, fallback rules, and the OIDC Model 2 SA discovery cache.
• `plugins/flytekit-spark/tests/test_databricks_token.py`: refactored to call into the strategy via select_auth while preserving the multi-tenant PAT scenarios from #3394.
• `plugins/flytekit-spark/tests/test_connector.py`: updated DatabricksJobMetadata constructions for the new fields.
• `plugins/flytekit-spark/README.md`: full Authentication section, env-var matrix, RBAC manifest for OIDC Model 2, migration guide from PAT.
Zero workflow-code changes
A core constraint of this PR is that existing Databricks workflows keep working without edits. Operators flip auth modes by setting connector-level env vars; task authors only touch DatabricksV2 fields if a single workflow needs to diverge from the connector default.
How was this patch tested?
Unit tests
$ cd plugins/flytekit-spark
$ pytest tests/test_databricks_auth.py tests/test_databricks_token.py tests/test_connector.py
============================ 100 passed in 3.04s ============================
The full plugin suite (pytest tests/ --ignore=tests/test_remote_register.py) shows 119 passed. The 5 failures in test_wf.py and test_pyspark_transformers.py are a pre-existing PySpark + Java 18+ compatibility issue (Exception: getSubject is not supported) on the local dev machine running Java 25; they pass on upstream CI which uses Java 11/17. None of those tests touch the files in this PR.
End-to-end on a development EKS cluster
Each auth mode was exercised against a real Databricks workspace by patching the connector deployment with an image built from this branch:
• `pat`: per-namespace databricks-token k8s secret. Existing path.
• `oauth_m2m`: per-namespace databricks-oauth k8s secret with client_id and client_secret keys, federation against a Databricks Service Principal.
• oidc_federation Model 2: per-workflow-namespace ServiceAccount annotated with flyte.org/databricks-enabled, flyte.org/databricks-client-id, flyte.org/databricks-audience. Connector minted a JWT via Kubernetes TokenRequest, exchanged it for a Databricks bearer token, and the resulting Jobs run in the Databricks console was attributed to the per-namespace Service Principal (verifying Unity Catalog tenancy is preserved).
Pre-commit
$ pre-commit run --files <the seven changed files>
ruff..................Passed
ruff-format...........Passed
codespell.............Passed
pydoclint.............Passed
pydoclint-errors-baseline.txt was not modified by this PR.
Setup process
The plugin README's Authentication section now documents the full operator setup, including the RBAC manifest for OIDC Model 2 and a migration guide from PAT. No infra changes are required for existing PAT users; the connector defaults still resolve PAT first.
Screenshots
n/a
Check all the applicable boxes
• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.
Related PRs
• #3394: original multi-tenant PAT support. This …
flyteorg/flytekit