https://flyte.org logo
d

Dylan Wilder

04/29/2022, 8:44 PM
how can i get workflow metadata at runtime these days? project, domain, workflow, task, version etc
p

Prafulla Mahindrakar

05/02/2022, 11:24 AM
you can get this using the get workflow admin api. eg : following command get this info
Copy code
flytectl get workflow -p flytesnacks -d development   core.flyte_basics.lp.go_greet --latest -o yaml
d

Dylan Wilder

05/02/2022, 1:40 PM
probably not the pattern we want to use from a task though right? anyway to grab from flytekit?
like the current context
related question: is it possible to inject env vars into workflows at launch time
found this. is it meant to be public?
s

Samhita Alla

05/02/2022, 2:18 PM
It should be accessible.
d

Dylan Wilder

05/02/2022, 2:19 PM
where? it used to be
flytekit.configuration.internal.DOMAIN
p

Prafulla Mahindrakar

05/02/2022, 2:39 PM
@Dylan Wilder this should be available to the user pod through these env var’s
Copy code
FLYTE_INTERNAL_EXECUTION_WORKFLOW:  flytesnacks:development:<http://example.wf|example.wf>
      FLYTE_INTERNAL_EXECUTION_ID:        a2r6nshjwvt8x54lp4c2
      FLYTE_INTERNAL_EXECUTION_PROJECT:   flytesnacks
      FLYTE_INTERNAL_EXECUTION_DOMAIN:    development
      FLYTE_ATTEMPT_NUMBER:               0
      FLYTE_INTERNAL_TASK_PROJECT:        flytesnacks
      FLYTE_INTERNAL_TASK_DOMAIN:         development
      FLYTE_INTERNAL_TASK_NAME:           example.generate_normal_df
      FLYTE_INTERNAL_TASK_VERSION:        O_YLNJX3tLNuJdNrP4WbdQ==
      FLYTE_INTERNAL_PROJECT:             flytesnacks
      FLYTE_INTERNAL_DOMAIN:              development
      FLYTE_INTERNAL_NAME:                example.generate_normal_df
      FLYTE_INTERNAL_VERSION:             O_YLNJX3tLNuJdNrP4WbdQ==
Every pod created by flyte would have these env variables which you can use for detecting the runtime workflow meta
re :
Copy code
found this. is it meant to be public?
This is used during registration and not consumed elsewhere
d

Dylan Wilder

05/02/2022, 3:00 PM
awesome! that's what i was looking for
k

Ketan (kumare3)

05/02/2022, 3:03 PM
@Dylan Wilder these are internal and meant to change, as we are working on compressing them. But which ones do you want
Most have been added to the context now
d

Dylan Wilder

05/02/2022, 3:04 PM
need, project, domain, workflow, version, exec id
feel like these should be part of some public api, i had expected to see them in the flyte context so that probably makes sense
many, many use cases but • need to differentiate prod /non-prod resource management • need to associate resources with specific production metadata
can i get more detail on the diff between EXECUTION* and TASK* vars
EXECUTION is the value of the parent workflow?
p

Prafulla Mahindrakar

05/02/2022, 4:37 PM
Cc : @Yee
y

Yee

05/02/2022, 4:43 PM
the difference is that it’s technically possible to launch a task in project a, under project b
so execution in this case would be b, but task_ would be a
d

Dylan Wilder

05/02/2022, 5:00 PM
and what about
FLYTE_INTERNAL_*
also, are any of these available at registration?
y

Yee

05/02/2022, 5:42 PM
what were you thinking (wrt registration)?
the short answer is no… because registration is a two step process. the definition of the task (and all other entities) is compiled with those placeholders. the task template also includes the env vars.
only when those protobufs are sent up to admin, are the placeholders replaced.
i think the public api that we add under context will end up using those
INTERNAL_
environment variables yes. the container plugin adds them.
g

Gigi

05/11/2022, 8:48 PM
We use the
FLYTE_INTERNAL_EXECUTION_DOMAIN
env var currently within tasks/workflows to configure environment specific external service urls
k

Ketan (kumare3)

05/11/2022, 9:38 PM
@Gigi I think we should starting using all of this from the context
👍 1
we want to have the flexibility to modify
INTERNAL
to improve performance etc
g

Gigi

05/11/2022, 10:02 PM
It's not currently in context, right? When it's there we would do something like
flytekit.current_context().domain
or something?
k

Ketan (kumare3)

05/11/2022, 10:36 PM
yup
that would be perfect
cc @Yee?
y

Yee

05/11/2022, 10:37 PM
yeah we can make a ticket for this
can you put the interface you want to see here @Dylan Wilder? https://github.com/flyteorg/flyte/issues/2490
5 Views