Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

has anyone else encountered something similar? Is there a way to provision more shared memory to a specific task?

you’re right we should probably address this along with gpus

Thanks for the quick reply. We added a `pod_template_name` that volumes it in.
A feature request could be to control this via the resource requests :stuck_out_tongue:

yeah we’re seeing if there’s anything common between migs and shm

shm is basically set at as a <https://kubernetes.io/docs/concepts/storage/volumes/#emptydir|volume> (memory-backed `emptyDir` with an optional `sizeLimit`) and a corresponding volume mount to the primary container. i think the only thing it has in common with migs is that it requires an update to the pod spec. :sweat_smile:

well, and maybe how it’s specified by the user.

in flytekit? there might also validations we need to do where shm size limit has to be less than memory limit - shm usage counts towards container memory usage. and sizelimit is also optional - uses up to container memory limit.

Hey <@UNR3C6Y4T> <@U017K8AJBAN> <@U052L7XMLHL>

Did you guys solve it in any easy way or any example for pod_template for that?

I’m encountering it when working with DataLoaders :stuck_out_tongue:

there is an example here: <https://github.com/unionai-oss/llm-fine-tuning/blob/77c42f7b297aec8783720ce2dd14fe873cb55503/fine_tuning/llm_fine_tuning.py#L378|https://github.com/unionai-oss/llm-fine-tuning/blob/77c42f7b297aec8783720ce2dd14fe873cb55503/fine_tuning/llm_fine_tuning.py#L378>