Is it possible to have flyte deployment (I'm tryin...
# flyte-deployment
f
Is it possible to have flyte deployment (I'm trying to deploy flyte-binary helm chart) scoped to a single / limited number of namespaces? We have K8s cluster that has strict isolation policies and Flyte's behaviour of creating namespaces + having the need to have some
ClusterRole
s being present is a major blocker. I saw that for namespace's there's a
Copy code
configmap:
    namespace_config:
      namespace_mapping:
        template: <my single NS>
setting. I've tried to work around the
ClusteRole
by using namespace-level
Role
instead, but it still Flyte fails to start (CrashLookBackOff) due to:
Copy code
cannot create resource \"customresourcedefinitions\" in API group \"apiextensions.k8s.io\" at the cluster scope
Is there any way around that? 🤔
f
The crd I think needs a cluster role
Flyte can work on one namespace
f
So replacement of ClusterRole by Role should work?
f
you will also have to change the cli for starting propeller
i like the idea of having a single namespace mode available
a
@flat-waiter-82487 according to the templates, if you set this to
false
https://github.com/flyteorg/flyte/blob/13b3d82c94fb61047a49aa2093a2b17fb6a7c792/charts/flyte-binary/values.yaml#L368-L370 Helm won't create the ClusterRole I haven't tried any of this, but then you could create a Role and RoleBinding to Flyte's service account and the CRD's API group
f
@average-finland-92144 this is exactly what I did - I have used
rbac.create=false
and used my own
Role
/
RoleBinding
instead.
I've tried to work around the
ClusteRole
by using namespace-level
Role
instead
Once the Deployment starts in k8s, it wants to create the CRD at runtime and due to missing
ClusterRole
- it fails.
you will also have to change the cli for starting propeller
@freezing-airport-6809 - which one? I'm using
flyte-binary
helm chart
a
@flat-waiter-82487 with
flyte-core
is possible to disable CRD creation by Helm so you could have more control over the process: https://github.com/flyteorg/flyte/blob/13b3d82c94fb61047a49aa2093a2b17fb6a7c792/charts/flyte-core/templates/propeller/crds/flyteworkflow.yaml#L2 I think CRDs are always cluster-scoped but instances are, in the case of the
flyteworkflow
CRD, namespaced https://github.com/flyteorg/flyte/blob/13b3d82c94fb61047a49aa2093a2b17fb6a7c792/charts/flyte-core/templates/propeller/crds/flyteworkflow.yaml#L19
f
Yeah, I would prefer to use
flyte-binary
, since it's easier and more suitable to our needs. I also saw that there is no option to install CRD as a part of Helm Chart in
flyte-binary
😞
Will setting
Copy code
propeller:
      create-flyteworkflow-crd: false
have a desired effect here? https://github.com/flyteorg/flyte/blob/f7450303998094289cef2f8106c185eb8efc670a/charts/flyte-binary/templates/configmap.yaml#L43 I could create PR for that
a
it should have that effect, yes Making it configurable for flyte-binary would be a great contribution. Let me know if you need help with that. Thank you!
f
I've made changes as per your suggestions - anything else needed to get this merged?
@average-finland-92144 is there anything else needed for this PR or can you merge? 🙂
a
@flat-waiter-82487 I tested the chart and can confirm the expected behavior with the CRD creation. Thank you! Merged ✅