https://flyte.org logo
g

Greg Gydush

04/21/2022, 8:49 PM
I was using the following code before (can explain use case if helpful) and was wondering if there is an alternative with the recent config changes. My current code (prior to 0.32)
Copy code
from flytekit.configuration import internal

internal.VERSION.get()
Potential new way (after 0.32)
Copy code
from flytekit.configuration.file import ConfigEntry, LegacyConfigEntry

ConfigEntry(LegacyConfigEntry("internal", "version")).read()
Is this the correct way I should be getting workflow version?
y

Yee

04/21/2022, 9:38 PM
background on this… this setting is actually something we’ve been trying to get away from for a while. in the past, we used to run the compilation step of turning python code into flyte tasks and workflows from within the container image itself.
we have moved away from that because we feel users typically don’t care for that… they’d rather just run the compilation step from within their working environment (like locally on their laptops). especially with the new
pyflyte run
experience it makes even less sense.
this setting was a way that we could inject the version of the task/workflow (basically the git sha) into the image itself. we did this by providing a special build script that added this tag.
basically the ‘version’ here is supposed to correspond to the version that these flyte entities will be registered to the Flyte backend with.
which means this is not really necessary at all at “compile time”. only when you send those objects to Admin do you need to provide the version
so in the new experience this is what we do… we just put in this value which is replaced by
flytectl
(or in the case of
pyflyte run
just overridden at grpc call time)
k

Ketan (kumare3)

04/21/2022, 11:06 PM
@Greg Gydush why do you want to get the version in the code?
g

Greg Gydush

04/21/2022, 11:17 PM
I see - appreciate the thorough background / explanation!
@Ketan (kumare3) I actually used that version to automatically set cache version for all tasks in a given workflow. We like having cache tied to workflow version (overridable by environment variable)
k

Ketan (kumare3)

04/21/2022, 11:19 PM
so you want basically every new version of git-sha to be a new cache version
g

Greg Gydush

04/21/2022, 11:19 PM
It’s a little more complicated than that, but kind of
We use more than git-sha to register (e.g., image hash, different strategy for fast registration)
Is what I suggested okay? Or no. I can also reach into our logic for naming to come up with the sha/image hash myself (rather than grab from current registered workflow)
k

Ketan (kumare3)

04/22/2022, 4:35 PM
what is your suggestion, i did not catch it
g

Greg Gydush

04/22/2022, 4:36 PM
Copy code
from flytekit.configuration.file import ConfigEntry, LegacyConfigEntry

ConfigEntry(LegacyConfigEntry("internal", "version")).read()
^ will this no longer work
y

Yee

04/22/2022, 5:20 PM
what do you mean?
that should continue to work for a while yes… but that is your own option
right? we removed that a while back.
but my question is… so long as you’re making your own option, why not just read from an env var where you need it, instead of routing the logic through flytekit config code
g

Greg Gydush

04/22/2022, 6:31 PM
FLYTE_INTERNAL_VERSION was something that flytekit provided? On versions before < 0.32
y

Yee

04/22/2022, 6:32 PM
it was something that flytekit used…
g

Greg Gydush

04/22/2022, 6:32 PM
Yes, but you’re saying it’s gone now?
y

Yee

04/22/2022, 6:32 PM
more accurately, it may have been used.
g

Greg Gydush

04/22/2022, 6:32 PM
It is not possible to get this now.. right?
but that is your own option
This was not my own option is what I’m saying. This was an internal variable that I’d still like to use. If I cannot, I will come up with an alternative (totally fine)
y

Yee

04/22/2022, 6:33 PM
even on older versions - if you ran
pyflyte package
and then
flytectl register
it would not have been used, even though the config object was still in the code.
g

Greg Gydush

04/22/2022, 6:34 PM
I think we’re just out of date @jeev (maybe we should change registration) - I’ll look at this closer later today or next week!
y

Yee

04/22/2022, 6:35 PM
yeah if possible, use an alternative…
happy to hop on a call as well
a

austin

04/26/2022, 9:54 PM
https://github.com/flyteorg/flyte/releases/tag/v1.0.0 <- is that typos in the OSS version? They contain references to
<http://flyte.lyft.com|flyte.lyft.com>
…? ( or is that on purpose ? )
7 Views