Hi everyone, I am trying to leverage pod templates...
# flyte-support
f
Hi everyone, I am trying to leverage pod templates, but it does not seem to work right. I have this pod template deployed in our cluster
Copy code
apiVersion: v1
kind: PodTemplate
metadata:
  name: flyte-burst-nodepool
template:
  spec:
    containers:
    - image: replace-me
      name: default
      resources:
        limits:
          cpu: "1"
          memory: 1Gi
        requests:
          cpu: 100m
          memory: 500Mi
    nodeSelector:
      poolname: burstpool
    tolerations:
    - effect: NoSchedule
      key: burstpool
      operator: Equal
      value: "true"
And then I try to use it like this
Copy code
@task(
    pod_template_name="flyte-burst-nodepool",
)
def my_burst_task(input_data: str) -> str:
    """
    Task that will run on the burst nodepool
    """
    processed_data = f"Processed: {input_data}"
    return processed_data
Which does actually work, the pod that gets created does have the correct nodeSelector and tolerations, but the resources are ignored completely. In the documentation, I've read that the container name should be specified either as default or primary, I tried both and neither works. I noticed that the container in the pods that are created by Flyte have names like this
Copy code
name: a8lhq75mv5q525x7d5ln-n0-0
Which references the execution id, so perhaps that is the issue but I couldn't find any mention on how to change the name of the container. Any help would be greatly appreciated.
f
Resources are controlled by Flyte
If you want to default them, you can do it in Flyte config
f
So it's impossible to set in pod templates?
f
Well nothing is impossible
It’s a code change
But maybe I am wrong let me talk to someone or look at code
f
Thank you. The reason this would be extremely helpful is that we have multiple types of nodes, and setting a single default value is not really possible because of the differences between the available resources on each node type. Also, configuring resources in code is not ideal because the developers write it themselves but are not familiar with the resources available in each environment, and each node type, which is why I would just like to give them a pod template per project, and configure it in a way that's optimal.
f
the idea of default resources is to have them set so that you can have some min / max
i think if you drop it, it should resolve to pod templates
f
I have these in my helm chart
Copy code
task_resources:
        limits:
          cpu: 1
          memory: 2Gi
          ephemeral-storage: 500Mi
      plugins:
        k8s:
          default-cpus: 50m
          default-memory: 256Mi
And I tried to get rid of that but it falled back to some default variables, I think it's these https://github.com/flyteorg/flyte/blob/b16ffd76934d690068db1265ac9907a278fba2ee/deployment/eks/flyte_helm_generated.yaml#L203-L213 Edit: When i get rid of the limits and defaults, it yields these resources and I could not find the source of these defaults
Copy code
resources:
      limits:
        cpu: "2"
        memory: 200Mi
      requests:
        cpu: "2"
        memory: 200Mi