Is there a way to configure Flyte to use different sets of resources for different kinds of workloads? We have a situation where user A submits long-running tasks and user B's short tasks don't get scheduled until user B's tasks finish. I see executionQueues mentioned in the docs but am not really understanding how to use them
10/22/2023, 11:46 PM
execution queues are for working with things like aws batch. i don’t think they’ll help in this scenario.
you can split them into multiple projects.
and then assign different namespace level k8s resourcequotas for those namespaces.
but that will permanently cap user A.
you don’t want to forcibly evict user a, that’s the worst outcome.
as user As tasks finish however you should be seeing user B’s tasks kick off. if you’re seeing that the user A workflow needs to run to completion before any of user b’s tasks start then something feels off.
10/23/2023, 1:06 AM
User A’s task is like a 35K map task. Could that explain this? I think different projects might be the solution for us
10/23/2023, 1:16 AM
i think you should try a different project.
easy enough to try, just re-register
map tasks shouldn’t inherently introduce issues in multi-tenancy beyond just the sheer number