sparse-pizza-79993
10/25/2023, 10:17 AMdevelopment
contains workflows currently being worked on, including ad-hoc registrations resulting from users running pyflyte run --remote …
; these are run against corresponding development environments of other apps.
2. staging
contains workflows that are versioned (incl. prerelease versions) for deployment against other corresponding staging environments. Essentially this would be a dedicated environment for QA without all the noise of development and to test rollout.
3. production
has the same versioned workflows as staging
for deployment against the corresponding official production environment.
The above is aspirational and I was really glad to see Flyte’s separation of domains per project; however I am missing some things to make this happen and would like to better understood how Flyte is intended to be used for this:
1. Development and Staging are fundamentally separate if all ways – they deploy separate code that is not “promoted” from one environment to the next. On the other hand, Production should run identical code without needing to rebuild/register the workflows after a particular version is cleared to be deployed to production. What is the appropriate way to deal with this? Or is it appropriate to simply reserialize and register the workflow again for the production domain?
2. As each environment is separate, there are separate secrets to be retrieved from an appropriate plugin; however I haven’t found a way to configure this at runtime, i.e. per domain without needing to hard-code the path to those secrets. Is there an idiomatic way to do this? Obviously hard-coding the secret path is not desirable as it would make it difficult to keep the code between staging and production identical (see previous point).
Any advice appreciatedbroad-train-34581
10/25/2023, 10:26 AMsparse-pizza-79993
10/25/2023, 10:31 AMSQLAlchemyTask
? There isn’t an accessible python runtime for that so one can’t access current_context()
broad-train-34581
10/25/2023, 10:43 AMflytekit.current_context().execution_id.domain
works outside tasks
if it doesn't there is an env var too, not sure if this is recommended though.
https://flyte-org.slack.com/archives/CP2HDHKE1/p1691687747543199?thread_ts=1691687330.613679&cid=CP2HDHKE1sparse-pizza-79993
10/25/2023, 11:01 AMsecret_connect_args
is the secrets are passed to the @task
annotation, which AFAICS is finalized at build-time…
I guess there isn’t a way to specify different secrets based on the domain which a workflow is deployed to?
Now that I think about it, even getting secrets from current_context()
still requires specifying those secret requests on the @task
beforehand anyway. I would need to load secrets for all environments into the task to have a switch based on flytekit.current_context().execution_id.domain
broad-train-34581
10/25/2023, 11:19 AMtall-lock-23197
sparse-pizza-79993
10/26/2023, 7:14 AMk8s
secrets, however I am using Hashicorp Vault secrets which means that either the group
or the key
will need to change between environments because they are not automatically separated per namespace.
I was just about to experiment with using something like the Vault Operator/Injector to create secrets automatically in each namespace and then reference those secrets from within Flyte as k8s
secrets… I’m sure that will work but it is unfortunate that I cannot seem to use the Vault plugin out of the box. I have also had other difficulties with it.tall-lock-23197
hallowed-mouse-14616
10/26/2023, 1:04 PMsparse-pizza-79993
10/26/2023, 1:23 PM