Eli Bixby11/30/2023, 12:02 PM
specified is? as long as it doesn't cause killing of the workflow, I think they should be able to work fine together
Dan Rammer (hamersaw)11/30/2023, 3:29 PM
Eli Bixby11/30/2023, 3:32 PM
Longer term I could see having job queues be a little bit more natively integrated as a way of providing priority/resource management, but don't have a clear sense of what that would look like.
Dan Rammer (hamersaw)11/30/2023, 3:36 PM
to kueue, wait for it to create a pod, etc. Propeller bases task status off pod / container status. So if kueue just leaves the Pod in
until it decides to execute there should be no issue. Very interested to see how this goes and happy to help if you run into issues!
Eli Bixby11/30/2023, 3:42 PM
field... So Flyte would need to change tasks to run as jobs and not pods.... no idea how involved that would be.
Correction: Looks like it will work just fine, just leave the pod in pending, provided you enable managing bare pods: https://kueue.sigs.k8s.io/docs/tasks/run_plain_pods/. We'll do a little write-up if we get it working
Dan Rammer (hamersaw)12/01/2023, 3:34 PM
David Espejo (he/him)12/06/2023, 3:35 PM
We'll do a little write-up if we get it workinghey @Eli Bixby, let us know, we can collaborate on this and post it on Flyte's blog too