Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

I assume <https://github.com/flyteorg/flyte/pull/5594|this> one?

<@U077V3Y43NC> can you judge whether yunikorn gang scheduling will be usable also with other plugins than ray like pytorch, mpi, tensorflow, … with your PR? :pray:

We use <https://github.com/kubernetes-sigs/scheduler-plugins/tree/531a1831bdda0bdad6057f7d00f3cafacdb93d86|scheduler plugins co-scheduling> but unfortunately observed some stability issues when there were many unschedulable pods due to resource constraints.

<@U04664Z7H37> Yes.
 Plugins in this PR add task group annotations to pods when a new job submit.
According to task group annotations,  pods under the job are scheduled with gang scheduling by Yunikorn.
So, gang scheduling is available when pods with task group annotations.

Yes.
Application ID and queue name are also important.
when defining partition and queue configuration in Yunikorn to control resource, queue priority and preemption.