:wave: we launched a workflow on the UI setting ma...
# flyte-support
g
👋 we launched a workflow on the UI setting max parallelism and the Flyte UI in the execution shows the
Parallelism
attribute as
5
but the workflow executed with more than 5 tasks in parallel. Are we misunderstanding the parallelism attribute or is there a bug here?
cc @polite-wire-1044 if you want to share more context
p
Hm, that pretty much captures it. The test code I wrote is:
Copy code
@task
def simple_task(idx: int) -> str:
    time.sleep(30)
    return f"task{idx}"
@dynamic
def run_tasks(n: int) -> List[str]:
    out = []
    for i in range(n):
        out.append(simple_task(i))
    return out

@workflow
def cttx_labeling() -> List[str]:
    return run_tasks(10)
m
are they all marked as Running or just Queued or Unknown state or are there actually more than 5 pods running in k8s
g
looking at our k8s dashboard I do see that 10 pods were in the
Running
phase concurrently
p
Yes, they were all immediately marked as running
m
hmm odd, what i usually see is 5 tasks in dynamic task will be running and other 5 are set to queued or unknown
i did a test run on a sandbox with the above and looks like its doing what it was intended 🤔 here the 3rd
chunk_and_process_wf
is on hold
i set the max parallelism to 2
p
@gentle-night-59824 I wonder why this doesn’t work for us
g
I'm not sure either - let me try repro-ing this and I'll look into it
e
Hey all! We encountered the same issue with max parallelism set to 5 *by default. Where is the exact location to configure the
MaxParallelism
value in the Helm chart? Thanks!
t
@elegant-toddler-67101, you should be able to configure it via flytectl: https://flyte-org.slack.com/archives/CP2HDHKE1/p1687109097892189?thread_ts=1687103342.136149&cid=CP2HDHKE1.
e
Is it not supported in the helm chart?
t
I think it should be supported. Do you have a
flyte-admin-base-config
config map in your deployment?
You can add
maxParallelism
to the
flyteadmin
block, Ariel. https://github.com/flyteorg/flyte/blob/951a93080972bbcf85bc92ec54ac62e074be887f/charts/flyte-core/values-eks.yaml#L16
Copy code
flyteadmin:
   maxParallelism: 25
253 Views