https://flyte.org logo
Powered by
#ask-the-community
Title
# ask-the-community
t

Tarmily Wen

11/22/2022, 5:30 PM
Is there a way to automatically taint a few node pools to allow gpu access upon creation? Like this but through the values.yaml when helm installing? Also what should the key and value be for the resource tolerations?
y

Yee

11/22/2022, 6:29 PM
can you elaborate a bit?
what do you mean by automatically taint?
setting taints on node pools is not something that’s done by flyte. that will have to be done by whoever is typically in charge of setting up the aws/eks resources.
once the taints are set, the tolerations for the gpu resource type should just be whatever you set as the taint.
t

Tarmily Wen

11/22/2022, 10:33 PM
So I can set the tolerations on the node pools by doing this in values.yaml
Copy code
k8s:
    plugins:
      # -- Configuration section for all K8s specific plugins [Configuration structure](<https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/pluginmachinery/flytek8s/config>)
      k8s:
        default-env-vars: []
        #  DEFAULT_ENV_VAR: VALUE
        default-cpus: 100m
        default-memory: 100Mi
        resource-tolerations:
          - <http://nvidia.com/gpu|nvidia.com/gpu>:
            - key: "flyte/gpu"
              operator: "Equal"
              value: "dedicated"
              effect: "NoSchedule"
But in order for a node pool to have a taint that matches these tolerations, I would need to taint them after helm installation. Is there a way to avoid doing the manual step of looking up node pools and setting the taint? It would be ideal if the node pool creator would apply taints to a few node pools.
y

Yee

11/22/2022, 11:28 PM
how are you creating the node pools?
and yes that’s correct - it’s a separate step
but note that this is just text (well a map of text)
basically it’s saying, for tasks that use that resource, apply these tolerations.
if you’re using terraform or something to create the node pools i’m sure it’s possible to add the taints at creation time.
flyte itself does not create node pools, it merely uses them
t

Tarmily Wen

11/22/2022, 11:56 PM
okay i understand now. thank you. I thought flyte created node pools since there are some default nodes under the development namespace.
Powered by