Would there be any way to set up gang scheduling w...
# announcements
r
Would there be any way to set up gang scheduling within a project/domain currently? I've seen something similar for Kubeflow's training operators using https://github.com/volcano-sh/volcano and was wondering if it'd be possible to achieve for Flyte workflows. For context, we'd like to create user-specific projects and enforce resource quotas on them in K8s; however, if we do this, we'd like to try to avoid multiple queued workflows contending for resources and let pods associated w/ a specific workflow get scheduled together. Another example of this would be https://github.com/palantir/k8s-spark-scheduler
k
Yes you can change the scheduler to use
And you will have to use kube batch or yunikorn etc
r
Ahh that's great. Thank you for the references @Ketan (kumare3)
160 Views