faint-activity-87590
05/30/2023, 4:29 PMaverage-finland-92144
05/30/2023, 6:09 PMfaint-activity-87590
05/30/2023, 6:11 PMaverage-finland-92144
05/30/2023, 6:14 PMmax_parallellism
in the launch plan?faint-activity-87590
05/30/2023, 6:16 PMLaunchPlan.get_or_create(
name="soft-energy-launchplan",
workflow=wf,
max_parallelism=10,
notifications=[
Slack(
phases=[
WorkflowExecutionPhase.SUCCEEDED,
WorkflowExecutionPhase.FAILED,
WorkflowExecutionPhase.ABORTED,
WorkflowExecutionPhase.TIMED_OUT,
],
recipients_email=["does-not-matter-what-you-put-in-here"],
),
],
)
Yes like this and it made no difference. For some reason registering this lp and exeuting it doesnt even show me the parallelism value in the ui but 0, but thats another problem 😛tall-lock-23197
max_parallelism
value isn't being respected for tasks in a dynamic workflow. It should, however, work correctly for tasks in a simple workflow. @high-accountant-32689, is this something we need to support?faint-activity-87590
05/31/2023, 6:17 AMmax_parallelism
is being respected? Would be really nice to control this like in a normal workflowfaint-activity-87590
05/31/2023, 9:02 AMfrom flytekit import workflow, task, dynamic, LaunchPlan, Resources, FixedRate
import time
from datetime import timedelta
@task(requests=Resources(cpu="500m", mem="500Mi"))
def say_hello(input: str):
time.sleep(60)
print(f"Hello {input}")
@workflow
def wf():
counter = 50
for i in range(counter):
say_hello(input="World")
LaunchPlan.get_or_create(
name="p-launchplan",
workflow=wf,
max_parallelism=2,
schedule=FixedRate(duration=timedelta(minutes=3)),
)
Executing this and observing the pods in kubernetes showed that Flyte is only launching 2 pods at a time but does not wait until the pods are done. So it basically increments with 2 over time and still ends up with 50 running pods at some time. I really thought max_parallelism
reflects the amount of RUNNING pods at a time. Is this intended or am i missing something?tall-lock-23197
I really thoughtYeah, I think it should. How's that workflow working for you? You cannot use loop in a Flyte workflow.reflects the amount of RUNNING pods at a time.max_parallelism
millions-night-34157
07/05/2023, 3:19 PMflytectl register files
command is trying to execute the launch plan from the local environment itself.faint-activity-87590
07/05/2023, 3:28 PM