Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

I need to introduce a GPU node for some tasks in a workflow. I've <https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/productionizing/configure_use_gpus.html|read this> introduction and understand kubernetes taints &amp; tolerations, but don't understand the components of this yaml.

```plugins:
  k8s:
    resource-tolerations:
      - <http://nvidia.com/gpu|nvidia.com/gpu>:
        - key: "key1"
          operator: "Equal"
          value: "value1"
          effect: "NoSchedule"```


My flyte config
```    plugins:
      k8s:
        resource-tolerations:
          - <http://nvidia.com/gpu|nvidia.com/gpu>:
              - key: "gpu"
                operator: "Equal"
                value: "true"
                effect: "NoSchedule"
        inject-finalizer: true```
My nodegroup taints
```<http://nvidia.com/gpu|nvidia.com/gpu>	true	NoSchedule
gpu	true	NoSchedule```
Error in console
```12/15/2023 3:53:22 PM UTC task submitted to K8s

12/15/2023 3:53:22 PM UTC Unschedulable:0/5 nodes are available: 1 Insufficient memory, 1 node(s) had untolerated taint {dedicated: flyte}, 2 Insufficient cpu, 4 Insufficient <http://nvidia.com/gpu|nvidia.com/gpu>. preemption: 0/5 nodes are available: 1 Preemption is not helpful for scheduling, 4 No preemption victims found for incoming pod..```

where are you setting this config? are you using the flyte binary helm chart? cc <@U04H6UUE78B>

<@U0539B9794L> from the config you share, it looks like you have 2 taints set on the node but only one matching toleration. For the `key` you can use `"<http://nvidia.com/gpu|nvidia.com/gpu>"` which, BTW, I think is the resource type exposed by the NVIDIA driver, so the key is restrictive here.
Also, is that new node also tainted with `dedicated:flyte`?