Hey everyone. I'm preparing a large batch job (256 parallel tasks), I noticed this "max parallelism" under domain settings in flyte console. • What is this limit exactly? • How can I adjust it? Can't really find anything on it in the (legacy) docs.

You can adjust per run

Yeah I want to do more, but GKE in our region has no more gpus available 😄

Got it

You can adjust per run

but what is it?

Parallelism limits how many will run at once

how many tasks? but i can already see i got more than 25 running, despite the limit written there

Within one graph - how many nodes

i have a dynamic workflow, not map task

I think you can look into the docs here: https://www.union.ai/docs/v1/flyte/deployment/flyte-configuration/customizable_resources/#workflow-execution-configuration Or you can set this through launch plan or through

default_options

in workflow

LaunchPlan.create(..., max_parallelism=256)

from flytekit.core.options import Options
@workflow(
      default_options=Options(max_parallelism=100)
  )