Hi I have a question about `@dynamic` workflows. ...
# ask-the-community
b
Hi I have a question about
@dynamic
workflows. If I create a dynamic workflow that invokes a set of tasks and one of those tasks fails but all the others succeeds, would it try to invoke the same set of tasks if I retrigger that same dynamic workflow? My situation is the following: • I want to compile a dbt project to read all the nodes in the dbt DAG and translate them to flyte tasks • I do not want to reprocess earlier nodes in the DAG if one of the nodes fail.
j
It wont trigger the successful tasks if you enabled caching and inputs to the task wont change
b
Oh awesome. And that’s it I enable caching on both the workflow and the tasks?
j
i dont think workflow has caching, its on the task level
you also want to checkout the
failure_policy
on the workflow that might be relevant for individual task failures
for example this one
@workflow(failure_policy=WorkflowFailurePolicy.FAIL_AFTER_EXECUTABLE_NODES_COMPLETE)
allows all possible tasks to be completed and wont stop if one of the task fails in the workflow
b
Gotcha. The dynamic workflow is just a container for the tasks and when it generates what tasks it’s gonna run flyte will check if these tasks already ran with these inputs and then skip them if they were successful. That’s awesome
j
yup
b
Thanks @Jay Ganbat !
127 Views