Hi all, I am trying out the Databricks integration and was just want to verify that it requires Databricks Container Services enabled?

Cc @Kevin Su

yes, because it allows us to change the task image.

Thanks for the info, are you planning to use something like spark connect in the future for this instead?

yes, we was thinking to use it, but it’s still WIP, and it will only support 3.4.0+. we can support that after it’s done.

Thanks for the info!