Would there be any way to set up gang scheduling within a project/domain currently? I've seen something similar for Kubeflow's training operators using
https://github.com/volcano-sh/volcano and was wondering if it'd be possible to achieve for Flyte workflows.
For context, we'd like to create user-specific projects and enforce resource quotas on them in K8s; however, if we do this, we'd like to try to avoid multiple queued workflows contending for resources and let pods associated w/ a specific workflow get scheduled together. Another example of this would be
https://github.com/palantir/k8s-spark-scheduler