Can anyone in the community provide some rough metrics on the Blob Storage usage by Flyte?
total # of workflows, executions, tasks
is raw user data stored in the same blob storage?
and the existing amount of storage used.
Also will appreciate it a lot if someone can share experiences where the blob storage becomes the bottleneck/where failure happens.
05/19/2023, 4:23 AM
@Yubo Wang blob storage usage can grow quite a bit, but caching, passing references reduces usage quite a bit.
Most cloud blob stores have a expiration policies, which works great with Flyte for completed workflows are data is never reused that way - you can simply delete data after a period