Is there a way I can modify default task tolerations in the

Question

Is there a way I can modify default task tolerations in the helm charts values file <https github com flyteorg flyte blob master charts flyte values yaml>

Yee · Answer

what s the use case gpus

Yee · Answer

more specifically are you looking to add tolerations to everything or only to certain task types

Yee · Answer

for everything add this <https github com flyteorg flyteplugins blob 6b72fd6b8d0f172bd64d967f13ec4cade51dba7e go tasks pluginmachinery flytek8s config config go L88>

Yee · Answer

by resource add this <https github com flyteorg flyteplugins blob 6b72fd6b8d0f172bd64d967f13ec4cade51dba7e go tasks pluginmachinery flytek8s config config go L117>

Yee · Answer

<https github com flyteorg flyteplugins blob 6b72fd6b8d0f172bd64d967f13ec4cade51dba7e go tasks pluginmachinery flytek8s config config go L133> may also be relevant

Ketan (kumare3) · Answer

Also for every container you can add default tolerations

Ketan (kumare3) · Answer

We are working on adding a podTemplate eventually per namespace that will be used as the default template for all container tasks

Fabio Grätz · Answer

My use case we share a cluster with some other workloads an mlflow tracking server instance a webapp that uses the deployed models and I don t want these pods to use the expensive machines when no training is occurring thus the taint It is ok that all flyte pods by default have the toleration With the provided links I was able to configure a default toleation Thanks for the guidance and really nice that this can be customized in such a fine grained way

Fabio Grätz · Answer

Is there a way to specify tolerations also for Spark tasks

Yee · Answer

< Anmol> do you know this by chance tolerations for spark tasks

Ketan (kumare3) · Answer

< Fabio Grätz> there is a way to add tolerations for spark

Ketan (kumare3) · Answer

but the spark plugin does not support that today I think the spark plugin needs an upgrade IMO it should be something like this ``` task config=Spark pod template=Pod def foo ```

Fabio Grätz · Answer

Actually setting this up in the spark helm chart itself is absolutely valid I think Deploying the spark helm chart with a tolerations value fails atm but I will debug why

Fabio Grätz · Answer

Thanks for the hints

Fabio Grätz · Answer

< Emirhan Karagül>

Emirhan Karagül · Answer

Thanks for the update I think the `tolerations` value of spark operator helm chart doesn t work out of box I will try to configure that one more time tomorrow

Fabio Grätz · Answer

< katrina>

Fabio Grätz · Answer

Thread I mentioned in my dm