hi team, the documentation states that subworkflow...
# announcements
a
hi team, the documentation states that subworkflows execute within context of the parent workflows (here). What are the implications of that?
k
they will run on the same propeller, they have the same settings like project/domain/service_accounts and other settings like interruptible
a
the doc also says they'll have same parallelism, but isn't parallelism configured at task level?
k
no parallelism (node parallelism) is at the workflow level
a
are you referring to max_parallelism arg of LaunchPlan?
k
yes
a
Copy code
max_parallelism (Optional[int]) – Controls the maximum number of tasknodes that can be run in parallel for the entire workflow. This is useful to achieve fairness. Note: MapTasks are regarded as one unit, and parallelism/concurrency of MapTasks is independent from this.
do I read this right that it doesn't apply to maptasks? What about spark?
k
yes
it applies to spark “task” not to spark executors
a
got it, thanks
173 Views