ripe-smartphone-56353
09/13/2024, 1:42 PMjolly-florist-66728
09/13/2024, 1:43 PMmax_parallelism
setting. This can be configured at the platform level, per launch plan, or per execution.
1. Platform Default: Set a platform-wide default for maximum parallelism.
2. Launch Plan: Override the default for a specific launch plan using flytekit.LaunchPlan.get_or_create()
.
3. Execution: Override the default for a specific execution using pyflyte run --max-parallelism
or by setting it in the UI.
from flytekit import LaunchPlan
LaunchPlan.get_or_create(
name="my_cron_scheduled_lp",
workflow=my_workflow,
max_parallelism=10,
)
For more details, refer to the Flyte documentation.
Would you like more detail on any specific method?
Sources:
- Flyte Documentation on Concurrency vs Parallelism 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.ripe-smartphone-56353
09/13/2024, 1:44 PMjolly-florist-66728
09/13/2024, 1:44 PMmax_parallelism
attribute at the workflow level. This attribute limits the number of parallel tasks within a workflow execution.
from flytekit import LaunchPlan
LaunchPlan.get_or_create(
name="my_workflow_lp",
workflow=my_workflow,
max_parallelism=10,
)
Additionally, you can configure this setting at the platform level or for specific executions. For more details, refer to the Flyte documentation.
Would you like more detail on configuring this for specific executions or at the platform level?
Sources:
- Flyte Documentation on Concurrency vs Parallelism 👍 0 button 👎 1 button 💡 Hint: Mention @ancient-wolf-19325 in the thread for followups.