Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

Not exactly a flyte issue directly, but I notice I sometimes experience `DiskPressure` on my nodes in my GKE cluster. Anyone got some best practices that I can implement on how to mitigate this? Currently I just have increased the disk sizes of my nodes which obviously gives it more space, but is not necessarily a cost effective solution

Use ephemeral disks if available on the node

Disk pressure is unavoidable if you download data

Use flytefile and flytedir and dataframe streaming APIs 

I'll look into ephemeral disks.
Would DiskPressure also happen if I do make sure to delete the files before finishing a task?

It can right if you have more tasks on one node

Hey, the streaming APIs are exactly what I was looking at for another issue. Thanks <@UNZB4NW3S>  I think it will indeed also help with the disk pressure.