@ancient-wolf-19325 in union serverless, how can I'm getting an error with DataLoader workers running out of shared memory. Typically with a docker container I could do

docker run --shm-size=2g your_image

. How can I achieve a similar result on union serverless?

In Union Serverless (and Flyte generally), increasing shared memory for DataLoader workers (e.g., for PyTorch) is not directly configurable via a

docker run --shm-size

equivalent. Instead, shared memory is typically managed by mounting an

emptyDir

volume with

medium: Memory

at

/dev/shm

in the pod spec. In Flyte, this is usually handled by specifying a custom PodTemplate or, for some plugins (like PyTorch/Elastic), by using the

increase_shared_mem

option or a

shared_memory

parameter in the task decorator. However, as of now, Union Serverless does not expose a direct user-facing option to set shared memory size like

--shm-size

in Docker, and you cannot pass this as a resource in the

@task

decorator in serverless mode (source, source). If you are using the PyTorch or Elastic plugin, shared memory is added by default unless explicitly disabled, but the size is typically set to the node's allocatable memory and is not user-configurable in serverless. For other task types, there is currently no supported way to increase

/dev/shm

size in Union Serverless. You may need to contact Union support for a feature request or consider using BYOC (Bring Your Own Cluster) for more control (source, source). Would you like more detail or example code for BYOC or custom pod templates? Sources: - GitHub Issue: Make shared memory a first-class citizen - GitHub Issue: Improve flytekitplugins-kfpytorch user experience with default pod template and other reasonable defaults 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 will i be able to do this in union byoc

Yes, in Union BYOC (Bring Your Own Cluster), you can control shared memory by customizing the pod spec. You can specify a PodTemplate or use the

@task(shared_memory="2Gi")

decorator argument to mount a memory-backed volume at

/dev/shm

with your desired size, similar to

docker run --shm-size=2g

(source, source). This is not available in Union Serverless, but is supported in BYOC. Would you like a code example for setting this up? Sources: - GitHub Pull Request: Adds shared memory into extended resources - GitHub Issue: Make shared memory a first-class citizen 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.