agreeable-kitchen-44189
05/18/2022, 8:54 AMPythonTask.pre_execute()
, so user_params: ExecutionParameters
would also be available if that helps.thankful-minister-83577
thankful-minister-83577
thankful-minister-83577
thankful-minister-83577
thankful-minister-83577
thankful-minister-83577
thankful-minister-83577
freezing-airport-6809
agreeable-kitchen-44189
05/19/2022, 5:47 AMflytekit
dask
plugin at the moment (very similar to this Flyte issue). I thought it might be nice if the dask
cluster spins up its scheduler and worker nodes with the same image as the one that the Flyte task is running on (in the remote execution case).
As a side note, if we things are working, we would also like to upstream our changes and contribute the plugin to the flytekit
repository. I've currently based my work on the new (and experimental) dask operator.freezing-airport-6809
freezing-airport-6809
freezing-airport-6809
echoing-queen-96254
05/19/2022, 3:04 PMagreeable-kitchen-44189
05/19/2022, 5:46 PMflytekit
plugin, which is only a few lines of code) 👍
@echoing-queen-96254 Sorry for that, this should be the link: https://github.com/flyteorg/flyte/issues/427echoing-queen-96254
05/19/2022, 5:46 PMfreezing-airport-6809