:wave: we launched a workflow on the UI setting ma...
# ask-the-community
s
👋 we launched a workflow on the UI setting max parallelism and the Flyte UI in the execution shows the
Parallelism
attribute as
5
but the workflow executed with more than 5 tasks in parallel. Are we misunderstanding the parallelism attribute or is there a bug here?
cc @Seth Hendrickson if you want to share more context
s
Hm, that pretty much captures it. The test code I wrote is:
Copy code
@task
def simple_task(idx: int) -> str:
    time.sleep(30)
    return f"task{idx}"
@dynamic
def run_tasks(n: int) -> List[str]:
    out = []
    for i in range(n):
        out.append(simple_task(i))
    return out

@workflow
def cttx_labeling() -> List[str]:
    return run_tasks(10)
j
are they all marked as Running or just Queued or Unknown state or are there actually more than 5 pods running in k8s
s
looking at our k8s dashboard I do see that 10 pods were in the
Running
phase concurrently
s
Yes, they were all immediately marked as running
j
hmm odd, what i usually see is 5 tasks in dynamic task will be running and other 5 are set to queued or unknown
i did a test run on a sandbox with the above and looks like its doing what it was intended 🤔 here the 3rd
chunk_and_process_wf
is on hold
i set the max parallelism to 2
s
@Swarup Srinivasan I wonder why this doesn’t work for us
s
I'm not sure either - let me try repro-ing this and I'll look into it
a
Hey all! We encountered the same issue with max parallelism set to 5 *by default. Where is the exact location to configure the
MaxParallelism
value in the Helm chart? Thanks!
s
a
Is it not supported in the helm chart?
s
I think it should be supported. Do you have a
flyte-admin-base-config
config map in your deployment?
You can add
maxParallelism
to the
flyteadmin
block, Ariel. https://github.com/flyteorg/flyte/blob/951a93080972bbcf85bc92ec54ac62e074be887f/charts/flyte-core/values-eks.yaml#L16
Copy code
flyteadmin:
   maxParallelism: 25
134 Views