ancient-wolf-19325
07/08/2024, 11:59 AM# repo1/workflow_a.py
from flytekit import task, workflow
@task
def task_a():
print("Running task in workflow A")
return "output from A"
@workflow
def workflow_a() -> str:
return task_a()
Repo 2 (Workflow B):
# repo2/workflow_b.py
from flytekit import task, workflow
@task
def task_b(input_from_a: str):
print(f"Running task in workflow B with input: {input_from_a}")
return "output from B"
@workflow
def workflow_b(input_from_a: str) -> str:
return task_b(input_from_a=input_from_a)
Parent Workflow to Chain Them
You can create a parent workflow that chains these two workflows. This parent workflow can be defined in either of the repositories or a separate one.
# parent_workflow.py
from flytekit import workflow
from repo1.workflow_a import workflow_a
from repo2.workflow_b import workflow_b
@workflow
def parent_workflow():
output_a = workflow_a()
output_b = workflow_b(input_from_a=output_a)
return output_b
Running the Parent Workflow
To run the parent workflow on the Flyte cluster, you can use the pyflyte run command:
pyflyte run --remote parent_workflow.py parent_workflow
This setup ensures that workflow_b runs only if workflow_a completes successfully, and it uses the output of workflow_a as the input to workflow_b.
Would you like more details on any specific part of this setup?
Sources:
- Chaining Flyte entities
- Chaining Flyte entities (GitHub)
Slack Link
https://flyte-org.slack.com/archives/CP2HDHKE1/p1720439955.166889 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.