Can we use NFS instead of S3 in k8s cluster with Flyte

Question

Ketan (kumare3) · Answer

< illarion Disabled> you can but to communicate with PRopeller you have to use s3 equivalent

Ketan (kumare3) · Answer

so most data can be in NFS

illarion Disabled · Answer

< Ketan kumare3 > What do you mean by communicate with PRopeller

illarion Disabled · Answer

I am looking on this scheme and do not see S3 there

Ketan (kumare3) · Answer

illarion Disabled · Answer

< Ketan kumare3 > Am i right there is impossible to use NFS directly instead of S3 Minio GCS for cross task data flow

Ketan (kumare3) · Answer

It is not impossible

Ketan (kumare3) · Answer

Nobody has tried

Ketan (kumare3) · Answer

File system is an option

Ketan (kumare3) · Answer

Would you Be open to experiment and contribute

Ketan (kumare3) · Answer

If so < illarion Disabled> wait till Tuesday we are merging this

Ketan (kumare3) · Answer

All backend one binary one config and simple to test and even deploy

Ketan (kumare3) · Answer

And we would love to help you

Ketan (kumare3) · Answer

Just to explain backend supports local files <https github com flyteorg flytestdlib blob d0a5767d30272c065a075a283fd2ba563834baca storage stow store go L53|https github com flyteorg flytestdlib blob d0a5767d30272c065a075a283fd2ba563834baca storage stow store go L53>