Mike Zhong08/09/2022, 6:16 PM
volume in my
cluster as a shared mount for
. I have looked at this example and see the
being passed into the pod spec. I’m looking through these docs and see that you can create a
. So in theory, I can just create a PV and PVC in my cluster, include them in my pod spec, and attach the spec to a flyte task. But I noticed that PVCs are namespace specific and flyte uses the
namespace for tasks/workflows that are executing. Two questions.
1. Are PVCs the right solution here and if so, how can I dynamically create PVCs for my
s? Is this something flyte could be configured to do for us or would we be responsible for ensuring any referenced PVCs and PVs exist.
2. What other options are available for mounting shared persistent volumes to my flyte tasks?
SeungTaeKim08/10/2022, 12:52 AM
Yuvraj08/10/2022, 2:04 AM
, For creating PVC for each project-domain namespace like this https://github.com/flyteorg/flyte/blob/master/charts/flyte-core/values-eks.yaml#L360
@Mike Zhong There are two ways to mount volume in your flyte task,
• First is using pod plugin
• Second is propeller’s pod template https://docs.flyte.org/en/latest/deployment/cluster_config/general.html#using-default-k8s-podtemplates
Mike Zhong08/10/2022, 1:51 PM
is that we either need to know up-front what all the projects are, or we will have to constantly update our flyte deployment with new templates as projects are added. Not sure if that is an improvement over just using CI/CD and shell scripts to manually create the namespace PVCs as needed
Mike Zhong08/10/2022, 8:48 PM
block is optional, but if we define namespace PVCs there, we would need to add them as new projects are created and redeploy to keep things updated. I was able to seamlessly get EFS working with flyte tasks so that was nice, just trying to figure out the best way to grant and manage access to the PV from other projects
Mike Zhong08/10/2022, 11:45 PM
). Since the namespaces are
and PVCs exist per namespace, any project that wants to use an
backed PV would need the PVC created for its namespace. Please correct me if this is not correct