Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

Hi all, I am trying out the Databricks integration and was just want to verify that it requires Databricks Container Services enabled?

yes, because it allows us to change the task image.

Thanks for the info, are you planning to use something like <https://issues.apache.org/jira/browse/SPARK-39375|spark connect> in the future for this instead?

yes, we was thinking to use it, but it’s still WIP, and it will only support 3.4.0+. we can support that after it’s done.