@ancient-wolf-19325, How do I add custom toleration?

To add custom tolerations in Flyte, you can modify the

values.yaml

file of the Flyte Helm chart. According to this discussion, you can add tolerations under the

extraPodSpec

section in the

values.yaml

file. Here's an example:

deployment:
  extraPodSpec:
    tolerations:
      - key: "dedicated"
        operator: "Equal"
        value: "flyte"
        effect: "NoSchedule"

Would you like more detail on this process? Sources: - Could I add tolerations on the flyte-binary helm chart value 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325, toleration not being applied on the pods

If tolerations are not being applied to the pods, ensure that the configuration is correctly set in the

values.yaml

file of the Flyte Helm chart. According to this discussion, you can specify tolerations under the

default-tolerations

section in the

values.yaml

file. Here's an example:

configuration:
  inline:
    plugins:
      k8s:
        default-tolerations:
          - key: 'flyte.org/node-role'
            operator: 'Equal'
            value: 'worker'
            effect: 'NoSchedule'

Ensure that the node has the corresponding taint and that the Flyte deployment is using the correct configuration. Would you like more detail on troubleshooting this issue? Sources: - Hey guys Following deployment guide I got to a point with a 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 How do we define toleration in the ImageSpec

Defining tolerations directly in the

ImageSpec

is not explicitly mentioned in the provided data sources. However, tolerations can typically be set in the pod specification, which is part of the Kubernetes configuration rather than the image specification itself. You can define tolerations in the

PodTemplate

when configuring tasks in Flyte, as shown in this GitHub issue:

from flytekit import PodTemplate, task
from kubernetes.client import V1PodSpec, V1Toleration

@task(
    pod_template=PodTemplate(
        pod_spec=V1PodSpec(
            tolerations=[
                V1Toleration(
                    key="app",
                    operator="Equal",
                    value="flyte",
                    effect="NoSchedule",
                ),
            ],
        ),
    ),
)
def my_task() -> None:
    print("Task with tolerations")

Would you like more detail on how to apply this configuration? Sources: - BUG Spark Task tolerations not applied with PodTemplate 👍 1 button 👎 0 button 💡 Hint: Mention @ancient-wolf-19325 in the thread for followups.

@ancient-wolf-19325, I tried above code, I get the following error: ValueError: Invalid value for

containers

, must not be

None

The error indicates that the

containers

field in the

V1PodSpec

is not properly defined. According to this Flyte example, you need to ensure that the

containers

field is specified, even if it's an empty list, to allow Flyte to insert a placeholder primary container. Here's how you can modify the code:

from flytekit import PodTemplate, task
from kubernetes.client import V1PodSpec, V1Toleration, V1Container

@task(
    pod_template=PodTemplate(
        pod_spec=V1PodSpec(
            containers=[V1Container(name="primary")],  # Ensure containers list is not None
            tolerations=[
                V1Toleration(
                    key="app",
                    operator="Equal",
                    value="flyte",
                    effect="NoSchedule",
                ),
            ],
        ),
    ),
)
def my_task() -> None:
    print("Task with tolerations")

Would you like more detail on this solution? Sources: - FlyteSnacks Pod Example 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.