https://flyte.org
Join Slack
Menu
Home
Archive
Not exactly a flyte issue directly, but I notice I sometimes experience `DiskPressure` on my nodes i...
# flyte-support
g
Not exactly a flyte issue directly, but I notice I sometimes experience 
DiskPressure
on my nodes in my GKE cluster. Anyone got some best practices that I can implement on how to mitigate this? Currently I just have increased the disk sizes of my nodes which obviously gives it more space, but is not necessarily a cost effective solution
f
Use ephemeral disks if available on the node
Disk pressure is unavoidable if you download data
Use flytefile and flytedir and dataframe streaming APIs
g
What's the die streaming API?
I'll look into ephemeral disks. Would DiskPressure also happen if I do make sure to delete the files before finishing a task?
f
It can right if you have more tasks on one node
@gentle-tomato-480 updated
g
Hey, the streaming APIs are exactly what I was looking at for another issue. Thanks @freezing-airport-6809 I think it will indeed also help with the disk pressure.
f
yup
4 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by