New RunLLM response generated: >>>*# Question* Hello everyone, we are currently trying to leverage GPU nodes on Azure, and we have not managed to find a way how to apply tolerations to the tasks pods. We have added some configuration based on this documentation: https://docs.flyte.org/en/latest/user_guide/productionizing/configuring_access_to_gpus.html However, tolerations are still not being applied to the pods. This is our current configuration:

configuration:
    inline:
      task_resources:
        defaults:
          cpu: 500m
          memory: 1Gi
          <http://nvidia.com/gpu|nvidia.com/gpu>: "1"
        limits:
          cpu: 2
          memory: 2Gi
          <http://nvidia.com/gpu|nvidia.com/gpu>: "1"
      plugins:
        k8s:
          inject-finalizer: true
          default-memory: 200Gi
          default-cpus: "20"
          resource-tolerations:
            - gpu:
              - key: "gpu"
                operator: "Equal"
                value: "true"
                effect: "NoSchedule"
          gpu-resource-name: "<http://nvidia.com/gpu|nvidia.com/gpu>"
          default-node-selector:
            poolname: gpu

We can see it's present in the main Flyte pod config file but the only thing that is being applied to the task pod is the nodeSelector

nodeSelector:
    poolname: gpu

I really appreciate any help, thank you! # Answer It seems like the tolerations are not being applied correctly to your task pods. Based on the provided configuration and the documentation, the issue might be due to the incorrect structure of the

resource-tolerations

configuration. The

resource-tolerations

should be a map, not a list. Here's an example of how to correctly configure the tolerations:

configuration:
  inline:
    task_resources:
      defaults:
        cpu: 500m
        memory: 1Gi
        nvidia.com/gpu: "1"
      limits:
        cpu: 2
        memory: 2Gi
        nvidia.com/gpu: "1"
    plugins:
      k8s:
        inject-finalizer: true
        default-memory: 200Gi
        default-cpus: "20"
        resource-tolerations:
          nvidia.com/gpu:
            - key: "gpu"
              operator: "Equal"
              value: "true"
              effect: "NoSchedule"
        gpu-resource-name: "nvidia.com/gpu"
        default-node-selector:
          poolname: gpu

Ensure that the

resource-tolerations

section is correctly formatted as a map with the GPU resource name (

nvidia.com/gpu

) as the key. Would you like more details or further assistance on this configuration? Sources: - Configuring access to GPUs - Customizing task resources Slack Link https://flyte-org.slack.com/archives/CP2HDHKE1/p1718696835.248509 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.