https://flyte.org
Join Slack
Hi team, What is the best way to solve this use case I have a wf `A` with inputs `a,b,c` I have a...
# flyte-support
s
Hi team, What is the best way to solve this use case I have a wf 
A
with inputs 
a,b,c
I have another wf B which find out what the inputs should be to wf A and “invokes” it. My current model is as below: Are there better ways to handle this?
Copy code
@workflow
A:

@workflow
B:
result = B(..)
t
depends on what you mean by find out.
if the logic is static, you can use a combination of conditionals and call A inside B as a subworkflow
or you can write a dynamic task that invokes A
s
Can a dynamic task invoke another workflow? 🤔
t
sorry, missed this somehow
yeah,
it can call another subworkflow, or a launch plan, or another task, or another dynamic task.
(that last one tho… not really a good/common pattern)
s
Personally I treat tasks and workflows as interchangeable entities during orchestration. So
Copy code
Workflow X:
-  task A(): outputs a,b,c
-  task B(a,b,c)
Copy code
Workflow A()
- ...
- ...
outputs a,b,c

Workflow B(a,b,c)
...

Workflow X:
- workflow A(): outputs a,b,c
- workflow B(a,b,c)
s
Ah sounds good, thank you 🙏
Hi @thankful-minister-83577 - The above solution works great~
One more follow up question - how does dynamic task decide how many workflows to launch? If my dynamic task has input 100, is there a parameter which decides this parallelism?
t
there’s a max parallelism setting on the launch plan
but are you calling workflows directly or calling a launch plan?
s
Copy code
ground_truth_workflow = flytekit.LaunchPlan.get_or_create(
    name="ground_truth_workflow",
    workflow=GroundTruthOfflinePCPWorkFlow,
)
Dynamic tasks calls a launch plan.
Copy code
@flytekit.dynamic
def run:
     # Figures out number of iterations
     # ground_truth_workflow(..)
t
got it okay, thank you.
the launch plan that you call will have its own parallelism setting
iirc, these aren’t shared, so you can end up running more than perhaps you intended.
however all runs are still subject to the project/domain level resource constraints
s
the launch plan that you call will have its own parallelism setting
Oh nice, that’s great! 😅
160 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by