Can we use NFS instead of S3 in k8s cluster with F...
# announcementsi
illarion Disabled
04/06/2022, 11:29 PMCan we use NFS instead of S3 in k8s cluster with Flyte?
k
Ketan (kumare3)
04/07/2022, 4:08 AM
@illarion Disabled you can, but to communicate with PRopeller you have to use s3 (equivalent)
Ketan (kumare3)
04/07/2022, 4:09 AM
so most data can be in NFS
i
illarion Disabled
04/07/2022, 9:01 AM@Ketan (kumare3) What do you mean by “communicate with PRopeller”?
illarion Disabled
04/07/2022, 9:02 AMI am looking on this scheme and do not see S3 there
k
Ketan (kumare3)
04/07/2022, 4:24 PM
i
illarion Disabled
04/08/2022, 11:22 AM@Ketan (kumare3) Am i right - there is impossible to use NFS directly instead of S3/Minio/GCS for cross-task data flow? (https://docs.flyte.org/en/latest/concepts/data_management.html#divedeep-data-management )
k
Ketan (kumare3)
04/08/2022, 1:31 PM
It is not impossible
Ketan (kumare3)
04/08/2022, 1:31 PM
Nobody has tried
Ketan (kumare3)
04/08/2022, 1:31 PM
File system is an option
Ketan (kumare3)
04/08/2022, 1:31 PM
Would you Be open to experiment and contribute
Ketan (kumare3)
04/08/2022, 1:32 PM
If so @illarion Disabled wait till Tuesday- we are merging this https://github.com/flyteorg/flyte/pull/2265
Ketan (kumare3)
04/08/2022, 1:33 PM
All backend one binary, one config and simple to test and even deploy
Ketan (kumare3)
04/08/2022, 1:33 PM
And we would love to help you
Ketan (kumare3)
04/08/2022, 1:34 PM
Just to explain backend supports local files https://github.com/flyteorg/flytestdlib/blob/d0a5767d30272c065a075a283fd2ba563834baca/storage/stow_store.go#L53
165 Views