Hi all, I'm getting an OOM error and concerned the limits I'm attempting to provide aren't being taken. When I kubectl describe the pod that failed due to OOMKilled - I see that it says the pod has a memory limit of 2G. However, in the flyte-deployment-config.yaml that I deployed using helm, I have allocated for a memory limit of 8G. See attached images for screenshots of the limit and the description. None of the @tasks have any resource limits imposed. Any help would be much appreciated!

Hi Ruksana! I could be wrong because I'm new to Flyte myself, but I think the

task_resources

helm configuration lists the

defaults

that will be applied to a task unless they request something different. The

limits

in that config would be the maximum that the tasks are allowed to ask for, but they will use the defaults unless they request more. Try setting some requests for 4G or 8G on your task and see if it is allocated that. Or, if you all or most of your tasks will require more memory, set the default config higher.

@Neal Feierabend Interesting! For some reason, I thought they'd scale automatically if they needed more - but I can see why they wouldn't. I'll try this !