hi team, When using ShellTask, is there way to attach an ephemeral storage (emptyDir) to it ? The Sh...

adamant-sandwich-39749

10/14/2022, 9:03 AM

hi team, When using ShellTask, is there way to attach an ephemeral storage (emptyDir) to it ? The ShellTask needs some scratch space to download files.

tall-lock-23197

10/14/2022, 11:55 AM

You can set ephemeral storage as

ephemeral_storage

in the task resources. You can send requests to shell task like any other task.

tall-lock-23197

10/14/2022, 11:56 AM

ShellTask(…, requests=Resources(ephemeral_storage="1Gi"))

freezing-airport-6809

10/14/2022, 2:32 PM

Btw she’ll task can also become a pod, if you pass config=Pod(

adamant-sandwich-39749

10/14/2022, 3:46 PM

what is the mount point of the emptyDir inside the container ?

adamant-sandwich-39749

10/14/2022, 4:19 PM

tried adding ephemeral request, it does not add any emptyDir volume to the pod

freezing-airport-6809

10/14/2022, 5:08 PM

it does not

freezing-airport-6809

10/14/2022, 5:08 PM

you have to use podConfig to do it

freezing-airport-6809

10/14/2022, 5:08 PM

https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/kubernetes/pod/pod.html#sphx-glr-auto-integrations-kubernetes-pod-pod-py

freezing-airport-6809

10/14/2022, 5:09 PM

if you see ShellTask it can take a https://docs.flyte.org/projects/flytekit/en/latest/generated/flytekit.extras.tasks.shell.ShellTask.html#flytekit.extras.tasks.shell.ShellTask Taskconfig

freezing-airport-6809

10/14/2022, 5:09 PM

this is the same as pod task

freezing-airport-6809

10/14/2022, 5:09 PM

cc @tall-lock-23197

👍 1

154 Views

Open in Slack

Previous Next

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.