Hey I've installed flyte on GKE it was running fin...
# flyte-supporte
echoing-river-3611
03/29/2023, 12:00 PMHey I've installed flyte on GKE it was running fine till now but i wanted to increase ResourceQuota for development/ staging and production namespace so i changed the config for the same but suddenly flyte-admin pod sync-cluster-resource container is failing with this error.
Copy code
panic: Failed to sync cluster resources : no matches for kind "Role" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "RoleBinding" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "Role" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "RoleBinding" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "Role" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "RoleBinding" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "Role" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "RoleBinding" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "Role" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "RoleBinding" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "Role" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "RoleBinding" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "Role" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "RoleBinding" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "Role" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "RoleBinding" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "Role" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "RoleBinding" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>"
goroutine 1 [running]:
main.main()
/go/src/github.com/flyteorg/flyteadmin/cmd/main.go:13 +0x91echoing-river-3611
03/29/2023, 12:32 PMIf my understanding is correct why is it looking for
<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1><http://rbac.authorization.k8s.io/v1|rbac.authorization.k8s.io/v1>f
freezing-airport-6809
03/29/2023, 1:06 PM
Interesting is v1beta1 deprecated
freezing-airport-6809
03/29/2023, 1:06 PM
Cc @acceptable-policeman-57188 can you please TAL
a
average-finland-92144
03/29/2023, 2:46 PM@echoing-river-3611 what's your K8s cluster version?
`rbac.authorization.k8s.io/v1beta1`was deprecated starting K8s 1.22
https://cloud.google.com/kubernetes-engine/docs/deprecations/apis-1-22#rbac-resources-v122
a
acceptable-policeman-57188
03/29/2023, 5:09 PM
hey @echoing-river-3611 just to double check, are you using the boilerplate configs for cluster resource controller here or are you adding your own for managing roles and role bindings? I don't believe we've added any role/role binding resource management to the cluster resource controller in the boilerplate helm charts
e
echoing-river-3611
03/29/2023, 6:12 PMk8s version >=1.24, yes everything is same in helm chart i just added labels to deployment and changed resourcequota
echoing-river-3611
03/29/2023, 6:16 PM@average-finland-92144 i know it got deprecated and all roles and role-binding got created with
v1v1beta1a
acceptable-policeman-57188
03/29/2023, 7:01 PM
@echoing-river-3611 what version of the flyte helm chart are you installing?
e
echoing-river-3611
03/29/2023, 7:06 PM1.3.0
echoing-river-3611
03/29/2023, 7:29 PMalso one more question, does cluster-resource-manager only create spark, spark-role, spark-role-bindings once or update them ? reason being i forgot to add some annotations so i changed the templates section with annotations added in all three even if on doing helm upgrade it didn't updated the resources so i thought of deleting them if i deleted them maybe it will re-create but thats not the case either so i had to manually create those sa, role and role-binding in each [development, staging and production] namespace.
echoing-river-3611
03/29/2023, 7:31 PMon helm upgrade clusterresource-template configmap did got updated, but sa, role and role-bindings didn't got updated
a
acceptable-policeman-57188
03/29/2023, 7:37 PM
it should update, but you might need to restart the sync job
e
echoing-river-3611
03/30/2023, 5:30 AMokk i think i got the problem, the doc for k8s-plugin [spark] https://docs.flyte.org/en/latest/deployment/plugins/k8s/index.html#deployment-plugin-setup-k8s
it is still using v1beta1 in template section for spark-role and spark-role-bindings. 😅
a
acceptable-policeman-57188
03/30/2023, 9:04 PM
nice find! do you mind opening a PR to fix this in open source docs?
158 Views