https://flyte.org
Join Slack
how can i get workflow metadata at runtime these days? project, domain, workflow, task, version etc
# flytekit
f
how can i get workflow metadata at runtime these days? project, domain, workflow, task, version etc
i
you can get this using the get workflow admin api. eg : following command get this info
Copy code
flytectl get workflow -p flytesnacks -d development   core.flyte_basics.lp.go_greet --latest -o yaml
f
probably not the pattern we want to use from a task though right? anyway to grab from flytekit?
like the current context
related question: is it possible to inject env vars into workflows at launch time
found this. is it meant to be public?
t
It should be accessible.
f
where? it used to be 
flytekit.configuration.internal.DOMAIN
i
@famous-businessperson-24711 this should be available to the user pod through these env var’s
Copy code
FLYTE_INTERNAL_EXECUTION_WORKFLOW:  flytesnacks:development:<http://example.wf|example.wf>
      FLYTE_INTERNAL_EXECUTION_ID:        a2r6nshjwvt8x54lp4c2
      FLYTE_INTERNAL_EXECUTION_PROJECT:   flytesnacks
      FLYTE_INTERNAL_EXECUTION_DOMAIN:    development
      FLYTE_ATTEMPT_NUMBER:               0
      FLYTE_INTERNAL_TASK_PROJECT:        flytesnacks
      FLYTE_INTERNAL_TASK_DOMAIN:         development
      FLYTE_INTERNAL_TASK_NAME:           example.generate_normal_df
      FLYTE_INTERNAL_TASK_VERSION:        O_YLNJX3tLNuJdNrP4WbdQ==
      FLYTE_INTERNAL_PROJECT:             flytesnacks
      FLYTE_INTERNAL_DOMAIN:              development
      FLYTE_INTERNAL_NAME:                example.generate_normal_df
      FLYTE_INTERNAL_VERSION:             O_YLNJX3tLNuJdNrP4WbdQ==
Every pod created by flyte would have these env variables which you can use for detecting the runtime workflow meta
re :
Copy code
found this. is it meant to be public?
This is used during registration and not consumed elsewhere
f
awesome! that's what i was looking for
f
@famous-businessperson-24711 these are internal and meant to change, as we are working on compressing them. But which ones do you want
Most have been added to the context now
f
need, project, domain, workflow, version, exec id
feel like these should be part of some public api, i had expected to see them in the flyte context so that probably makes sense
many, many use cases but • need to differentiate prod /non-prod resource management • need to associate resources with specific production metadata
can i get more detail on the diff between EXECUTION* and TASK* vars
EXECUTION is the value of the parent workflow?
i
Cc : @thankful-minister-83577
t
the difference is that it’s technically possible to launch a task in project a, under project b
so execution in this case would be b, but task_ would be a
f
and what about 
FLYTE_INTERNAL_*
also, are any of these available at registration?
t
what were you thinking (wrt registration)?
the short answer is no… because registration is a two step process. the definition of the task (and all other entities) is compiled with those placeholders. the task template also includes the env vars.
only when those protobufs are sent up to admin, are the placeholders replaced.
i think the public api that we add under context will end up using those 
INTERNAL_
environment variables yes. the container plugin adds them.
b
We use the 
FLYTE_INTERNAL_EXECUTION_DOMAIN
env var currently within tasks/workflows to configure environment specific external service urls
f
@brainy-ocean-79995 I think we should starting using all of this from the context
👍 1
we want to have the flexibility to modify 
INTERNAL
to improve performance etc
b
It's not currently in context, right? When it's there we would do something like 
flytekit.current_context().domain
or something?
f
yup
that would be perfect
cc @thankful-minister-83577?
t
yeah we can make a ticket for this
can you put the interface you want to see here @famous-businessperson-24711? https://github.com/flyteorg/flyte/issues/2490
195 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by