Hi! Is there a way to get current image FQN (in c...
# announcements
b
Hi! Is there a way to get current image FQN (in case of a remote execution) from within a task? Specifically I would be interested in having that information in
PythonTask.pre_execute()
, so
user_params: ExecutionParameters
would also be available if that helps.
y
hey!
so no, not at this time, but this is something i’m pretty sure we can add
there’s a related issue for improving the sematics around that object
could you elaborate on the use-case a bit please?
you can always inject the name of the image, into the image itself if you need, as an environment variable.
in fact we used to do this… but it ended up being error prone and complicated so we worked around it
@Marc Paquette
👍 1
k
@Bernhard Stadlbauer would love to understand the usecase. and also file an issue. I think you can do this today, by adding the image as an environment variable. We did not want rely on Flyte internal vars
b
Thanks for the quick response @Yee @Ketan (kumare3)! The usecase is that I am working on a
flytekit
dask
plugin at the moment (very similar to this Flyte issue). I thought it might be nice if the
dask
cluster spins up its scheduler and worker nodes with the same image as the one that the Flyte task is running on (in the remote execution case). As a side note, if we things are working, we would also like to upstream our changes and contribute the plugin to the
flytekit
repository. I've currently based my work on the new (and experimental) dask operator.
k
Fantastic tbh, this would be best accomplished using a backend plugin
And for operators the plugins are easy to implement
Also will allow to make the UX great and keep complexity of the plugin in flytekit minimal
m
@Bernhard Stadlbauer Your link to the similar Flyte issue seems to be broken. Can you send the link again?
b
Ok, @Ketan (kumare3) I'll look into the backend plugin, don't have a time estimate yet though (especially as we have a working
flytekit
plugin, which is only a few lines of code) 👍 @Marc Paquette Sorry for that, this should be the link: https://github.com/flyteorg/flyte/issues/427
👍 2
m
@Bernhard Stadlbauer No worries. Thanks!
k
@Bernhard Stadlbauer that is perfect. We can help with the backend plugin. But if you have a flytekit plugin, and are open to sharing it, we would love to TAL at it
169 Views