Channels
#flytekit
Title
d
Dylan Wilder
04/29/2022, 8:44 PMhow can i get workflow metadata at runtime these days? project, domain, workflow, task, version etc
p
Prafulla Mahindrakar
05/02/2022, 11:24 AMyou can get this using the get workflow admin api.
eg : following command get this info
Copy code
flytectl get workflow -p flytesnacks -d development core.flyte_basics.lp.go_greet --latest -o yamld
Dylan Wilder
05/02/2022, 1:40 PMprobably not the pattern we want to use from a task though right? anyway to grab from flytekit?
like the current context
related question: is it possible to inject env vars into workflows at launch time
found this. is it meant to be public?
s
Samhita Alla
05/02/2022, 2:18 PM
It should be accessible.
d
Dylan Wilder
05/02/2022, 2:19 PMwhere? it used to be
flytekit.configuration.internal.DOMAINp
Prafulla Mahindrakar
05/02/2022, 2:39 PM@Dylan Wilder this should be available to the user pod through these env var’s
Copy code
FLYTE_INTERNAL_EXECUTION_WORKFLOW: flytesnacks:development:<http://example.wf|example.wf>
FLYTE_INTERNAL_EXECUTION_ID: a2r6nshjwvt8x54lp4c2
FLYTE_INTERNAL_EXECUTION_PROJECT: flytesnacks
FLYTE_INTERNAL_EXECUTION_DOMAIN: development
FLYTE_ATTEMPT_NUMBER: 0
FLYTE_INTERNAL_TASK_PROJECT: flytesnacks
FLYTE_INTERNAL_TASK_DOMAIN: development
FLYTE_INTERNAL_TASK_NAME: example.generate_normal_df
FLYTE_INTERNAL_TASK_VERSION: O_YLNJX3tLNuJdNrP4WbdQ==
FLYTE_INTERNAL_PROJECT: flytesnacks
FLYTE_INTERNAL_DOMAIN: development
FLYTE_INTERNAL_NAME: example.generate_normal_df
FLYTE_INTERNAL_VERSION: O_YLNJX3tLNuJdNrP4WbdQ==Every pod created by flyte would have these env variables which you can use for detecting the runtime workflow meta
re :
Copy code
found this. is it meant to be public?d
Dylan Wilder
05/02/2022, 3:00 PMawesome! that's what i was looking for
k
Ketan (kumare3)
05/02/2022, 3:03 PM
@Dylan Wilder these are internal and meant to change, as we are working on compressing them. But which ones do you want
Most have been added to the context now
d
Dylan Wilder
05/02/2022, 3:04 PMneed, project, domain, workflow, version, exec id
feel like these should be part of some public api, i had expected to see them in the flyte context so that probably makes sense
many, many use cases but
• need to differentiate prod /non-prod resource management
• need to associate resources with specific production metadata
can i get more detail on the diff between EXECUTION* and TASK* vars
EXECUTION is the value of the parent workflow?
p
Prafulla Mahindrakar
05/02/2022, 4:37 PMCc : @Yee
y
Yee
05/02/2022, 4:43 PM
the difference is that it’s technically possible to launch a task in project a, under project b
so execution in this case would be b, but task_ would be a
d
Dylan Wilder
05/02/2022, 5:00 PMand what about
FLYTE_INTERNAL_*also, are any of these available at registration?
y
Yee
05/02/2022, 5:42 PM
what were you thinking (wrt registration)?
the short answer is no… because registration is a two step process. the definition of the task (and all other entities) is compiled with those placeholders. the task template also includes the env vars.
only when those protobufs are sent up to admin, are the placeholders replaced.
i think the public api that we add under context will end up using those
INTERNAL_g
Gigi
05/11/2022, 8:48 PMWe use the
FLYTE_INTERNAL_EXECUTION_DOMAINk
Ketan (kumare3)
05/11/2022, 9:38 PM
@Gigi I think we should starting using all of this from the context
👍 1
we want to have the flexibility to modify
INTERNALg
Gigi
05/11/2022, 10:02 PMIt's not currently in context, right? When it's there we would do something like
flytekit.current_context().domaink
Ketan (kumare3)
05/11/2022, 10:36 PM
yup
that would be perfect
cc @Yee?
y
Yee
05/11/2022, 10:37 PM
yeah we can make a ticket for this
can you put the interface you want to see here @Dylan Wilder? https://github.com/flyteorg/flyte/issues/2490
5 Views