Channels
datahub-flyte
scipy-2023-workshop
flyte-school
agent-support-memverge
flyte-build
flyte-users-berlin
scipy-2023-sprint
auth
flyte-bazel
large-language-models
contribute
great-content
in-flyte-conversations
flyte-on-gcp
show-and-tell
flyte-github
linkedin-flyte
random
deployment
hacktoberfest-2023
feature-discussions
linen-test
flytelab
flytekit-java
integrations
ray-on-flyte
conference-talks
release
flyte-ui-ux
workflow-building-ui-proj
writing-w-sfloris
jobs
hacktoberfest-2022
torch-elastic
flyte-console
engineeringlabs
helsing-flyte
flyte-documentation
konan-integration
databricks-integration
ray-integration
wg-gpu-types
flytekit
ecosystem-unionml
scipy-2022-sprint
announcements
ask-the-community
flyte-deployment
introductions
events
Powered by
#announcements
Title
i
illarion Disabled
04/06/2022, 11:29 PM
Can we use NFS instead of S3 in k8s cluster with Flyte?
k
Ketan (kumare3)
04/07/2022, 4:08 AM
@illarion Disabled
you can, but to communicate with PRopeller you have to use s3 (equivalent)
so most data can be in NFS
i
illarion Disabled
04/07/2022, 9:01 AM
@Ketan (kumare3)
What do you mean by “communicate with PRopeller”?
I am looking on this scheme and do not see S3 there
k
Ketan (kumare3)
04/07/2022, 4:24 PM
https://docs.flyte.org/en/latest/concepts/data_management.html#divedeep-data-management
👍 1
i
illarion Disabled
04/08/2022, 11:22 AM
@Ketan (kumare3)
Am i right - there is impossible to use NFS directly instead of S3/Minio/GCS for cross-task data flow? (
https://docs.flyte.org/en/latest/concepts/data_management.html#divedeep-data-management
)
k
Ketan (kumare3)
04/08/2022, 1:31 PM
It is not impossible
Nobody has tried
File system is an option
Would you Be open to experiment and contribute
If so
@illarion Disabled
wait till Tuesday- we are merging this
https://github.com/flyteorg/flyte/pull/2265
All backend one binary, one config and simple to test and even deploy
And we would love to help you
Just to explain backend supports local files
https://github.com/flyteorg/flytestdlib/blob/d0a5767d30272c065a075a283fd2ba563834baca/storage/stow_store.go#L53
5 Views
Post