https://flyte.org logo
Powered by
#flyte-deployment
Title
# flyte-deployment
m

Matheus Moreno

06/30/2022, 7:04 PM
Hi, everyone! We're finally deploying Flyte into our production environment, but there are some things that I have to configure before that. For instance, I noticed that projects have their own namespaces, such as <project>-development, <project>-staging and <project>-production. However, right now we only have one cluster on our GKE, and we isolate our systems using namespaces. I was wondering: is it possible to configure hierarchical namespaces in Flyte, so that all namespaces are inside a subnamespace "flyte" inside a namespace "ml-prod", for instance? Or, in a worst case scenario, is it possible to just let everything live in the "flyte" subnamespace?
(By the way, if anyone could help me with the deployment process, it would be great! I'm kinda confused by the need of a SSL certificate and the Ingress bit of the documentation. We're deploying on GCP, in a cluster with other projects/namespaces.)
Nevermind the SSL/Ingress, just realized those are configured for the namespace we use! So the problem continues to be a reliable way to define namespaces.
y

Yee

06/30/2022, 7:53 PM
yeah that’s this bit right?
somehow this makes its way into the cluster resource controller i think - the part of admin that runs against every k8s cluster and configures it
cc @katrina for when she’s back tuesday
👀 1
and oh, yeah, no, you can let everything live in one namespace… but not 
flyte
name it something else.
keep the flyte ns for the flyte stuff
m

Matheus Moreno

06/30/2022, 8:00 PM
so if I override the YAML configuration for, for instance, "template: flyte-projects", everything (dev, stage, prod) will "point" to this same ns, right?
y

Yee

06/30/2022, 8:00 PM
yup
m

Matheus Moreno

06/30/2022, 8:00 PM
perfect! thank you
y

Yee

06/30/2022, 8:00 PM
you may need quotes
but yeah
m

Matheus Moreno

06/30/2022, 8:10 PM
yeah, I'll add it
another question: in the tutorial, it says "<RELEASE-NAME> to be used as prefix for ssl certificate secretName". We already have a SSL certificate and an ingress controller configured. What should I put on the release name?
y

Yee

06/30/2022, 9:03 PM
oops
link me?
m

Matheus Moreno

06/30/2022, 9:07 PM
y

Yee

06/30/2022, 9:09 PM
looking up release.name i don’t see that much
m

Matheus Moreno

06/30/2022, 9:18 PM
oh ok, so it's just for a name
y

Yee

06/30/2022, 9:19 PM
i think so yeah… it’s just part of helm
m

Matheus Moreno

06/30/2022, 9:47 PM
Last question! What does cluster_resource_manager does, exactly? As I said before, this cluster does have other applications, and is not dedicated entirely for Flyte. So does it make sense to allow Flyte to alter cluster resources?
Or is it only executed on a namespace level?
y

Yee

06/30/2022, 10:19 PM
hmm yeah. we do promote that (dedicated clusters) but understand the reality of things. the cluster resource manager is just a bit that goes through the clusters (admin can be linked to multiple and round robin requests) and makes sure things are applied. by things, i mean this collection of yamls.
like when you tell flytectl to tell Admin that some project/domain can only have x cpu… something has to then tell all the clusters that that’s the case.
the cluster resource controller is the bit that does that
see multi cluster if you need that one day.
m

Matheus Moreno

07/01/2022, 12:38 AM
ohhh so it's for multi clusters? can I still configure individual project attributes without it, if I only have one cluster?
y

Yee

07/01/2022, 1:19 AM
yes
it’ll work for just one
a

Andrew Achkar

07/05/2022, 7:41 PM
Just to piggy back on this question, I am curious what are valid variables that can go in the namespace_config template string. Could one have a template like 
"{{ namespace }}-{{ project }}-{{ domain }}"
?
k

katrina

07/05/2022, 7:44 PM
hey @Andrew Achkar what is namespace supposed to represent here? for namespace templating the only variables permitted currently are 
{{ project }}
and 
{{ domain }}
a

Andrew Achkar

07/05/2022, 7:46 PM
I was thinking of the parent namespace. So say you deploy flyte under flyte namespace, then you’d get 
flyte-<project>-<domain>
but if you instead deploy in a test namespace like 
aachkar-flyte
then 
aachkar-flyte-<project>-<domain>
I suppose we could achieve something like this using helm
Copy code
namespace_config:
    namespace_mapping:
      template: "{{ .Release.Namespace }}-{{ project }}-{{ domain }}
k

katrina

07/05/2022, 10:22 PM
yeah that could work!
7 Views
Powered by