Hi community! I'm trying to run a dummy python job on a 🧱 job cluster (_new_cluster_) using

pyflyte

like this:

pyflyte run -i <corporate_docker_registry>/<prefix>/flyte-dbx-demo:0.0.1 --remote --destination-dir . dbx_example_job_cluster.py my_databricks_job

Job successfully submitted to Databricks, but the cluster creation failed because the IMAGE could not be pulled from our corporate docker registry (due to missing credentials). Analyzed the audit logs and found that the docker image config looks like this (_basic_auth_ block is missing):

"docker_image": {
  "url": "<corporate_docker_registry>/<prefix>/flyte-dbx-demo:0.0.1",
}

I'm assuming that if we add

imagePullSecrets

to our service account as described here, Flyte will pass the Docker credentials to Databricks in job definition like this:

"docker_image": {
  "url": "<corporate_docker_registry>/<prefix>/flyte-dbx-demo:0.0.1",
  "basic_auth": {
    "username": <user>,
    "password": <token>
  }
}

Can you please confirm? (Please note that I'm only experiencing this issue when trying to run a job on a _new_cluster,_ I was able to successfully complete a job on an _existing__cluster.) cc @Evan Sadler @Kevin Su

sorry, I didn’t know we can pass credentials to databricks job, so for now we won’t pass it to the dbx. I’ll add it. Contributions are welcome

All right, thanks for the quick feedback!

To workaround it, you can push image to the public registry, like docker hub.

That's not an option for us, we must use corporate docker registry.

I see, I’ll update it ASAP

Thanks Kevin!

thank you @Kevin Su - I have seen this change: https://github.com/flyteorg/flyte/pull/3788 I presume we need to wait for the next release. Actually we are on Flyte

1.5.0

, any chance to get this change backported?

@Robert Ambrus could you help me test it? https://github.com/flyteorg/flyteplugins/pull/364. I’ve built an image.

pingsutw/flytepropeller:47f81950416273150c4547f6c9cb0caef61f0eec

. btw, you have to mount the imagePullSecrets to propeller first.

Ok, thanks @Kevin Su! I'm working on a POC to integrate Flyte with Databricks, the Flyte services are not managed by me, but let me discuss it with the team who is responsible for that.

Hi Kevin, Did the pull request above (364) for passing the image credentials to dbx during cluster creation get merged? I see that the pull request is still in draft mode in the old repo and did not find that change incorporated in the mono repo where all the flyteplugins code was moved after 10/09