Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

Has anyone tried running Dask together with Flyte?

<@U01FB6HPVPB> great timing, infact folks from Pachama do this and they are kind enough to upstream the plugin
<https://github.com/flyteorg/flyteplugins/pull/275|https://github.com/flyteorg/flyteplugins/pull/275>


<@U01FB6HPVPB>
Hi from over here at Pachama :slightly_smiling_face:

Happy to share the approach we're using at the moment (with a `flytekit` "pure python" plugin), until the work on the backend plugin is finished.
Also feel free to send me a DM if you have any further questions

I'll try to get a PR out sometime this week for the flytekit plugin, <@UNZB4NW3S> I might do this in a separate repository, as the backend plugin will soon also have a `flytekit` PR?

<@U03C1MJQ892> also very interested in what a pure python plug-in might look like!

<@U03BT8P2XGA> you should be able to do everything in pure python right - with the right service account?

<@U01FB6HPVPB> <@U03BT8P2XGA> <@UNZB4NW3S>

Sorry that this took a little longer, but I've packaged up the initial `flytekit` `dask` code now into <https://github.com/bstadlbauer/flytekitplugins-dask|this repository>.

I've also released it to PyPi, so you should be able to install it with
```pip install bstadlbauer.flytekitplugins-dask```
Let me know if there are any issues, feature requests, etc. :slightly_smiling_face: