https://flyte.org logo
Join Slack
Powered by
I was using the following code before (can explain...
# flytekit
g
I was using the following code before (can explain use case if helpful) and was wondering if there is an alternative with the recent config changes. My current code (prior to 0.32)
Copy code
from flytekit.configuration import internal

internal.VERSION.get()
Potential new way (after 0.32)
Copy code
from flytekit.configuration.file import ConfigEntry, LegacyConfigEntry

ConfigEntry(LegacyConfigEntry("internal", "version")).read()
Is this the correct way I should be getting workflow version?
y
background on this… this setting is actually something we’ve been trying to get away from for a while. in the past, we used to run the compilation step of turning python code into flyte tasks and workflows from within the container image itself.
we have moved away from that because we feel users typically don’t care for that… they’d rather just run the compilation step from within their working environment (like locally on their laptops). especially with the new 
pyflyte run
experience it makes even less sense.
this setting was a way that we could inject the version of the task/workflow (basically the git sha) into the image itself. we did this by providing a special build script that added this tag.
basically the ‘version’ here is supposed to correspond to the version that these flyte entities will be registered to the Flyte backend with.
which means this is not really necessary at all at “compile time”. only when you send those objects to Admin do you need to provide the version
so in the new experience this is what we do… we just put in this value which is replaced by 
flytectl
(or in the case of 
pyflyte run
just overridden at grpc call time)
k
@Greg Gydush why do you want to get the version in the code?
g
I see - appreciate the thorough background / explanation!
@Ketan (kumare3) I actually used that version to automatically set cache version for all tasks in a given workflow. We like having cache tied to workflow version (overridable by environment variable)
k
so you want basically every new version of git-sha to be a new cache version
g
It’s a little more complicated than that, but kind of
We use more than git-sha to register (e.g., image hash, different strategy for fast registration)
Is what I suggested okay? Or no. I can also reach into our logic for naming to come up with the sha/image hash myself (rather than grab from current registered workflow)
k
what is your suggestion, i did not catch it
g
Copy code
from flytekit.configuration.file import ConfigEntry, LegacyConfigEntry

ConfigEntry(LegacyConfigEntry("internal", "version")).read()
^ will this no longer work
y
what do you mean?
that should continue to work for a while yes… but that is your own option
right? we removed that a while back.
but my question is… so long as you’re making your own option, why not just read from an env var where you need it, instead of routing the logic through flytekit config code
g
FLYTE_INTERNAL_VERSION was something that flytekit provided? On versions before < 0.32
y
it was something that flytekit used…
g
Yes, but you’re saying it’s gone now?
y
more accurately, it may have been used.
g
It is not possible to get this now.. right?
but that is your own option
This was not my own option is what I’m saying. This was an internal variable that I’d still like to use. If I cannot, I will come up with an alternative (totally fine)
y
even on older versions - if you ran 
pyflyte package
and then 
flytectl register
it would not have been used, even though the config object was still in the code.
g
I think we’re just out of date @jeev (maybe we should change registration) - I’ll look at this closer later today or next week!
y
yeah if possible, use an alternative…
happy to hop on a call as well
a
https://github.com/flyteorg/flyte/releases/tag/v1.0.0 <- is that typos in the OSS version? They contain references to 
<http://flyte.lyft.com|flyte.lyft.com>
…? ( or is that on purpose ? )
167 Views
Open in Slack
PreviousNext
Powered by