https://flyte.org logo
#ask-the-community
Title
# ask-the-community
k

Klemens Kasseroller

12/22/2023, 10:27 AM
Hi, I am trying to enable the dask plugin in my flyte deployment following the documentation and get the following error (in the flyte pod logs):
Copy code
W1222 10:08:49.441894       7 reflector.go:324] pkg/mod/k8s.io/client-go@v0.24.1/tools/cache/reflector.go:167: failed to list *v1.DaskJob: daskjobs.kubernetes.dask.org is forbidden: User "system:serviceaccount:flyte:flyte-backend-flyte-binary" cannot list resource "daskjobs" in API group "kubernetes.dask.org" at the cluster scope                                                                                                   E1222 10:08:49.441919       7 reflector.go:138] pkg/mod/k8s.io/client-go@v0.24.1/tools/cache/reflector.go:167: Failed to watch *v1.DaskJob: failed to list *v1.DaskJob: daskjobs.kubernetes.dask.org is forbidden: User "system:serviceaccount:flyte:flyte-backend-flyte-binary" cannot list resource "daskjobs" in API group "kubernetes.dask.org" at the cluster scope
Steps: Install dask operator with the helm chart (chart:dask-kubernetes-operator-2023.10.0, app-version:2022.4.1), update values, deploy flyte-binary helm chart (chartflyte binary v1.9.0,app version1.16.0) Does anyone have an idea what could be the problem? Thanks
b

Bernhard Stadlbauer

12/22/2023, 11:02 AM
I do Seems like your flyte service account does not have the permissions to look at dask’s custom resources. You’d probably wanna have a
ClusterRole
which you can then bind your used service account. The Cluster role should provide access to those resource, so something along the lines of:
Copy code
apiVersion: <http://rbac.authorization.k8s.io/v1|rbac.authorization.k8s.io/v1>
kind: ClusterRole
metadata:
  name: dask-dask-kubernetes-operator-role-cluster
  labels:
    <http://app.kubernetes.io/managed-by|app.kubernetes.io/managed-by>: Helm
  annotations:
    <http://meta.helm.sh/release-name|meta.helm.sh/release-name>: dask
    <http://meta.helm.sh/release-namespace|meta.helm.sh/release-namespace>: dask
rules:
  - verbs:
      - list
      - watch
    apiGroups:
      - <http://apiextensions.k8s.io|apiextensions.k8s.io>
    resources:
      - customresourcedefinitions
  - verbs:
      - get
      - list
      - watch
      - patch
      - create
      - delete
    apiGroups:
      - <http://kubernetes.dask.org|kubernetes.dask.org>
    resources:
      - daskclusters
      - daskworkergroups
      - daskjobs
      - daskjobs/status
      - daskautoscalers
      - daskworkergroups/scale
YMMV with which permissions you need
k

Klemens Kasseroller

12/22/2023, 11:23 AM
Works now, thanks a lot! 🙂
4 Views