Hey I've installed flyte on GKE it was running fin...
# ask-the-community
h
Hey I've installed flyte on GKE it was running fine till now but i wanted to increase ResourceQuota for development/ staging and production namespace so i changed the config for the same but suddenly flyte-admin pod sync-cluster-resource container is failing with this error.
Copy code
panic: Failed to sync cluster resources : no matches for kind "Role" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "RoleBinding" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "Role" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "RoleBinding" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "Role" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "RoleBinding" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "Role" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "RoleBinding" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "Role" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "RoleBinding" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "Role" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "RoleBinding" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "Role" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "RoleBinding" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "Role" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "RoleBinding" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "Role" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>", no matches for kind "RoleBinding" in version "<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>"

goroutine 1 [running]:
main.main()
/go/src/github.com/flyteorg/flyteadmin/cmd/main.go:13 +0x91
If my understanding is correct why is it looking for
<http://rbac.authorization.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1>
when in helm chart we explicitly mentioned that if cluster support
<http://rbac.authorization.k8s.io/v1|rbac.authorization.k8s.io/v1>
use that is this hardcoded in the code itself?
k
Interesting is v1beta1 deprecated
Cc @katrina can you please TAL
d
@HIMANSHU JOSHI what's your K8s cluster version? `rbac.authorization.k8s.io/v1beta1`was deprecated starting K8s 1.22 https://cloud.google.com/kubernetes-engine/docs/deprecations/apis-1-22#rbac-resources-v122
k
hey @HIMANSHU JOSHI just to double check, are you using the boilerplate configs for cluster resource controller here or are you adding your own for managing roles and role bindings? I don't believe we've added any role/role binding resource management to the cluster resource controller in the boilerplate helm charts
h
k8s version >=1.24, yes everything is same in helm chart i just added labels to deployment and changed resourcequota
@David Espejo (he/him) i know it got deprecated and all roles and role-binding got created with
v1
version but somehow flyteadmin is trying to find
v1beta1
version
k
@HIMANSHU JOSHI what version of the flyte helm chart are you installing?
h
1.3.0
also one more question, does cluster-resource-manager only create spark, spark-role, spark-role-bindings once or update them ? reason being i forgot to add some annotations so i changed the templates section with annotations added in all three even if on doing helm upgrade it didn't updated the resources so i thought of deleting them if i deleted them maybe it will re-create but thats not the case either so i had to manually create those sa, role and role-binding in each [development, staging and production] namespace.
on helm upgrade clusterresource-template configmap did got updated, but sa, role and role-bindings didn't got updated
k
it should update, but you might need to restart the sync job
h
okk i think i got the problem, the doc for k8s-plugin [spark] https://docs.flyte.org/en/latest/deployment/plugins/k8s/index.html#deployment-plugin-setup-k8s it is still using v1beta1 in template section for spark-role and spark-role-bindings. 😅
k
nice find! do you mind opening a PR to fix this in open source docs?
101 Views