jolly-florist-66728
11/13/2024, 10:20 AMmaxParallelism
only controls how many tasks within a single workflow can run concurrently, so it doesn’t solve our problem of limiting the total number of running workflows per project.
I’m considering using task resource limits (CPU, GPU, memory) to control the amount of resources per task as a workaround. We could manually adjust these limits for each project. However, I’d love to hear if there are better strategies or best practices for this scenario.
Also, is there a way to set a default timeout for every task globally, unless the user explicitly specifies a different timeout with @task(timeout=timedelta(hours=1))
?
Thanks in advance for any advice! 🙏
# Answer
To limit the number of workflows running simultaneously per project, you can consider setting project-level quotas. According to Flyte's discussion forum, you can define project quotas manually, which can help manage the number of concurrent workflows and prevent unexpected high costs. This approach is more aligned with your requirement than using maxParallelism
, which is task-specific.
For setting a default timeout for tasks globally, Flyte does not provide a direct way to set a global default timeout. You would need to specify the timeout for each task individually using the @task
decorator. However, you can manage task timeouts through configuration settings as mentioned in Flyte's documentation.
Would you like more details on implementing project-level quotas or managing task timeouts?
Sources:
- Flyte Discussion on Project Concurrency
Slack Link
https://.slack.com/archives/CP2HDHKE1/p1731493211.595029 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.