silly-refrigerator-21057
02/16/2023, 1:26 PMdefaultIamRole
(and ideally `projectQuotaCpu`/`projectQuotaMemory` , if that's still possible) to the pods that run in dev/stag/prod namespaces across all projects. To do this, I'm using the configuration.inline.cluster_resources
entry (see prod values).
When I deploy the chart with these values, I can see that these exist under 010-inline-config.yaml
in the flyte-backend-flyte-binary-config configmap, but I don't think these values are picked up by the namespaces. For instance, I would expect the default sa in each namespace (dev/stag/prod) to have a role-arn
annotation, but it doesn't.
Have I missed something very obvious?silly-refrigerator-21057
02/16/2023, 1:33 PMthankful-minister-83577
thankful-minister-83577
thankful-minister-83577
thankful-minister-83577
spark
service account.thankful-minister-83577
default
so that it also configures the default service accountsilly-refrigerator-21057
02/16/2023, 8:00 PMthankful-minister-83577
thankful-minister-83577
thankful-minister-83577
thankful-minister-83577
thankful-minister-83577
042_def_service_account.yaml: |
apiVersion: v1
kind: ServiceAccount
metadata:
name: default
namespace: '{{ namespace }}'
annotations:
<http://eks.amazonaws.com/role-arn|eks.amazonaws.com/role-arn>: '{{ defaultIamRole }}'
silly-refrigerator-21057
02/16/2023, 10:15 PMsilly-refrigerator-21057
02/17/2023, 11:35 AMconfiguration:
...
inline:
cluster_resources:
customData:
- production:
- defaultIamRole:
value: "<IAM_ROLE_ARN>"
- staging:
- defaultIamRole:
value: "<IAM_ROLE_ARN>"
- development:
- defaultIamRole:
value: "<IAM_ROLE_ARN>"
clusterResourceTemplates:
inline:
001_namespace.yaml: |
apiVersion: v1
kind: Namespace
metadata:
name: "{{ namespace }}"
042_def_service_account.yaml: |
apiVersion: v1
kind: ServiceAccount
metadata:
name: default
namespace: '{{ namespace }}'
annotations:
<http://eks.amazonaws.com/role-arn|eks.amazonaws.com/role-arn>: '{{ defaultIamRole }}'
I found that I needed to add 001_namespace.yaml
or else the project specific namespaces were not created. However, the default sa in each namespace does not have that role-arn annotation.
I can see that the flyte-backend-flyte-binary-cluster-resource-templates
config map does correctly contain the inline cluster resource templates. I can also see that the the cluster_resources
are contained in a file at /etc/flyte/config.d/010-inline-config.yaml
on the main flyte pod.
So it seems like everything is in place, but the annotation is not added to the default sa? Moreoever, it's not clear that the 042_def_service_account.yaml
does anything. If I edit the name and update the helm chart, there is no such service account created in any namespace. I'm all out of ideas at this point ๐silly-refrigerator-21057
02/20/2023, 8:06 AMsilly-refrigerator-21057
02/20/2023, 2:03 PMrbac:
extraRules:
- apiGroups:
- ""
resources:
- serviceaccounts
verbs:
- create
- get
- list
- patch
- update
- apiGroups:
- <http://rbac.authorization.k8s.io|rbac.authorization.k8s.io>
resources:
- rolebindings
- roles
verbs:
- create
- get
- list
- patch
- update
Simple fix in the end but took some time to figure out ๐thankful-minister-83577
thankful-minister-83577
thankful-minister-83577
thankful-minister-83577
powerful-gold-59386
02/20/2023, 5:58 PM