https://flyte.org
Join Slack
Is there a way to configure the scheduler or flyte in some way to only allow at most N instances of ...
# flyte-support
n
Is there a way to configure the scheduler or flyte in some way to only allow at most N instances of a certain workflow/task to run at the same time in a namespace?
f
Yup
You don’t have to configure scheduler
You can do that per workflow or per project - call max-parallelism
If you want to limit core use - resource quotas
n
mostly lets say I have a workflow making unsafe updates to a database and I just want to ensure that we only ever have one of these workflows running at a time.
Controls the maximum number of task nodes that can be run in parallel for the entire workflow. This is useful to achieve fairness.
this sounds like its for the tasks in the workflow? Or can it also control how many parallel instances of the workflow itself
f
That is not supported today
n
ah ok thanks.
f
But this is great, we would probably add this as a feature
As Flyte has resource manager
n
as a feature req, it would also be good to have an option to also be able to do this across namespaces, in case I've got a shared resource that both my dev/prod/wtv workflows are hitting
g
@numerous-hamburger-7178 mind creating a issue here [flyte-core]
u
f
yes, please add the details
again this is not prioritized, but good to have it in the the set of things to do
👍 1
171 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by