Is there a way to configure Flyte to use different sets of r

Question

Is there a way to configure Flyte to use different sets of resources for different kinds of workloads We have a situation where user A submits long running tasks and user B s short tasks don t get scheduled until user B s tasks finish I see <https github com flyteorg flyteadmin blob 6a64f00315f8ffeb0472ae96cbc2031b338c5840 flyteadmin config yaml L97 L106|executionQueues> mentioned in the docs but am not really understanding how to use them

Yee · Answer

execution queues are for working with things like aws batch i don t think they ll help in this scenario

Yee · Answer

you can split them into multiple projects

Yee · Answer

and then assign different namespace level k8s resourcequotas for those namespaces

Yee · Answer

but that will permanently cap user A

Yee · Answer

you don t want to forcibly evict user a that s the worst outcome

Yee · Answer

as user As tasks finish however you should be seeing user B s tasks kick off if you re seeing that the user A workflow needs to run to completion before any of user b s tasks start then something feels off

Gopal Vashishtha · Answer

User A s task is like a 35K map task Could that explain this I think different projects might be the solution for us

Yee · Answer

i think you should try a different project

Yee · Answer

easy enough to try just re register

Yee · Answer

map tasks shouldn t inherently introduce issues in multi tenancy beyond just the sheer number