acoustic-carpenter-78188
08/07/2023, 6:28 PMmaptasks
use their own concurrency configuration to set parallelism. By default, every subtask is executed in parallel. This can be confusing when integrating into an existing workflow where a user would like N tasks to execute simultaneously. If a maptask
is included in this workflow-level parallelism then N-1
tasks will execute in addition to the maptask
, where an additional M
(concurrency configuration of the maptask
) tasks will execute.
Goal: What should the final outcome look like, ideally?
maptasks
should be default configurable to respect the parallelism of the parent workflow.
Describe alternatives you've considered
Leaving it as is can be confusing and result in unintended behavior and / or difficulty achieving the desired level of parallelism in a workflow execution.
Propose: Link/Inline OR Additional context
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyte