Thread
#ask-the-community
    Mike Zhong

    Mike Zhong

    1 month ago
    Hello flyte team. I have a question regarding using persistent volumes (PVs) and persistent volume claims (PVCs) with flyte tasks and workflows. More specifically, I’m interested in using an AWS
    efs
    volume in my
    eks
    cluster as a shared mount for
    ReadWriteMany
    . I have looked at this example and see the
    V1Volume
    being passed into the pod spec. I’m looking through these docs and see that you can create a
    V1Volume
    from a
    persistent_volume_claim
    . So in theory, I can just create a PV and PVC in my cluster, include them in my pod spec, and attach the spec to a flyte task. But I noticed that PVCs are namespace specific and flyte uses the
    project-domain
    namespace for tasks/workflows that are executing. Two questions. 1. Are PVCs the right solution here and if so, how can I dynamically create PVCs for my
    project-domain
    s? Is this something flyte could be configured to do for us or would we be responsible for ensuring any referenced PVCs and PVs exist. 2. What other options are available for mounting shared persistent volumes to my flyte tasks?
    s

    SeungTaeKim

    1 month ago
    I have considered exactly the same problem in my team. here are some solution I do. 1. you can create pvc per namespace using IaC platform like Terraform, Pulumi. a. if you are using IaC , then you don't have to create each PVC per namespace manually. b. or you should make shell script to build all PVC per namespace at once. 2. how about using other path in V1Volume? you can use NFS or hostpath to access some specific volume. these are also be provided in V1Volume (kubernetes client, indeed)
    y

    Yuvraj

    1 month ago
    @SeungTaeKim @Mike Zhong You guys can use
    cluster_resource_manager
    , For creating PVC for each project-domain namespace like this https://github.com/flyteorg/flyte/blob/master/charts/flyte-core/values-eks.yaml#L360 @Mike Zhong There are two ways to mount volume in your flyte task, • First is using pod plugin • Second is propeller’s pod template https://docs.flyte.org/en/latest/deployment/cluster_config/general.html#using-default-k8s-podtemplates
    Ketan (kumare3)

    Ketan (kumare3)

    1 month ago
    Cc @Smriti Satyan docs issue please - I think we should have an example of using Efs - cc @Samhita Alla
    Mike Zhong

    Mike Zhong

    1 month ago
    Thanks for the replies, the pods plugin is what I have been using but I will definitely take a look at the propeller pod’s template and cluster_resource_manager
    The issue I see with defining namespace specific objects like PVCs in the
    cluster_resource_manager
    is that we either need to know up-front what all the projects are, or we will have to constantly update our flyte deployment with new templates as projects are added. Not sure if that is an improvement over just using CI/CD and shell scripts to manually create the namespace PVCs as needed
    Ketan (kumare3)

    Ketan (kumare3)

    1 month ago
    I don't think it is, it should be optional
    Would love ideas
    Mike Zhong

    Mike Zhong

    1 month ago
    the
    cluster_resource_manager
    block is optional, but if we define namespace PVCs there, we would need to add them as new projects are created and redeploy to keep things updated. I was able to seamlessly get EFS working with flyte tasks so that was nice, just trying to figure out the best way to grant and manage access to the PV from other projects
    Ketan (kumare3)

    Ketan (kumare3)

    1 month ago
    other projects?
    Mike Zhong

    Mike Zhong

    1 month ago
    other
    flyte
    projects (
    -p
    ). Since the namespaces are
    {project}-{domain}
    and PVCs exist per namespace, any project that wants to use an
    efs
    backed PV would need the PVC created for its namespace. Please correct me if this is not correct
    Ketan (kumare3)

    Ketan (kumare3)

    1 month ago
    hmm that is true. But, if you are using EFS just create PVC in every namespace and then use podtemplates to mount them. Now to provision them you can use ClusterResourceManager, or you can use flytectl (gitops) style to create namesapces and manage resources in that namespaces through your regular CI/CD tooling. and you can have a centralized way to onboard onto Flyte - Creating a project has to go through a central way
    We can discuss more about this if you want