https://flyte.org logo
Powered by
#ask-the-community
Title
# ask-the-community
v

varsha Parthasarathy

12/07/2022, 7:23 PM
Hi team, What is the best way to solve this use case I have a wf 
A
with inputs 
a,b,c
I have another wf B which find out what the inputs should be to wf A and “invokes” it. My current model is as below: Are there better ways to handle this?
Copy code
@workflow
A:

@workflow
B:
result = B(..)
y

Yee

12/07/2022, 7:37 PM
depends on what you mean by find out.
if the logic is static, you can use a combination of conditionals and call A inside B as a subworkflow
or you can write a dynamic task that invokes A
v

varsha Parthasarathy

12/07/2022, 7:44 PM
Can a dynamic task invoke another workflow? 🤔
y

Yee

12/07/2022, 8:38 PM
sorry, missed this somehow
yeah,
it can call another subworkflow, or a launch plan, or another task, or another dynamic task.
(that last one tho… not really a good/common pattern)
b

Babis Kiosidis

12/08/2022, 1:50 PM
Personally I treat tasks and workflows as interchangeable entities during orchestration. So
Copy code
Workflow X:
-  task A(): outputs a,b,c
-  task B(a,b,c)
Copy code
Workflow A()
- ...
- ...
outputs a,b,c

Workflow B(a,b,c)
...

Workflow X:
- workflow A(): outputs a,b,c
- workflow B(a,b,c)
v

varsha Parthasarathy

12/08/2022, 4:49 PM
Ah sounds good, thank you 🙏
Hi @Yee - The above solution works great~
One more follow up question - how does dynamic task decide how many workflows to launch? If my dynamic task has input 100, is there a parameter which decides this parallelism?
y

Yee

12/13/2022, 1:47 AM
there’s a max parallelism setting on the launch plan
but are you calling workflows directly or calling a launch plan?
v

varsha Parthasarathy

12/13/2022, 2:06 AM
Copy code
ground_truth_workflow = flytekit.LaunchPlan.get_or_create(
    name="ground_truth_workflow",
    workflow=GroundTruthOfflinePCPWorkFlow,
)
Dynamic tasks calls a launch plan.
Copy code
@flytekit.dynamic
def run:
     # Figures out number of iterations
     # ground_truth_workflow(..)
y

Yee

12/13/2022, 2:10 AM
got it okay, thank you.
the launch plan that you call will have its own parallelism setting
iirc, these aren’t shared, so you can end up running more than perhaps you intended.
however all runs are still subject to the project/domain level resource constraints
v

varsha Parthasarathy

12/13/2022, 2:12 AM
the launch plan that you call will have its own parallelism setting
Oh nice, that’s great! 😅
5 Views
Powered by