Has anyone tried using Kueue with Flyte?

Also interested in this. I have seen discussion in here before about it

I’ll get back to you tonight, busy atm, sorry 🙂

Thanks for the quick reply, no worries!

Hey @gentle-tomato-480 @worried-airplane-87065, I still need to merge this PR in flytekit so that one can set the queue name annotation in the

@task

decorator. Merging this PR requires a flyteidl release. I've been running a flytekit version with this change for a while and flyte with kueue works together well I'd say. I don't really use any of the queuing features of kueue but rather it's integration with the provisioning request api which works really well with gke. The only issue I came across is this issue with the GKE cluster autoscaler when you try to use provisioning requests with ephemeral volume claim templates.

I think we were planning a release next week.

is it ok to release a beta flyteidl for Fabio getting review in flytekit? I can do the beta release cc @thankful-minister-83577 @average-finland-92144

From my side we can also wait until next week but thank you 🙂

Yeah, also willing to wait for an official release before trying it all out

just wanted to say I'd be really interested in this integration as well 😄

Note that the PRs I contributed are not an integration with Kueue but a way to set labels/annotations in the

@task

decorator. With that, using flyte with kueue boils down to configuring a queue and setting its name in the task decorator as an annotation. No further coupling between flyte and kueue is needed.

makes sense! Thanks!