Geert
07/07/2023, 11:55 AM@dynamic
def base_workflow(config: Config):
for i in [1..10]:
task_a = a(i=i)
task_b = b(i=i)
task_c = c(i=i)
task_a >> task_b >> task_c
This all starts nicely 10x in parallel (as desired), but as soon as one of the tasks fail (lets say, task_a fails for i=3), they all abort.
Is there a Flyte-native way to allow i=3 to fail, but let the others complete?Maarten de Jong
07/07/2023, 12:03 PM@workflow(failure_policy=WorkflowFailurePolicy.FAIL_AFTER_EXECUTABLE_NODES_COMPLETE)
Geert
07/07/2023, 12:08 PM@dynamic
doesn’t have that field, but maybe I can find a way to nest a workflow or something to use it.Maarten de Jong
07/07/2023, 12:09 PMGeert
07/07/2023, 12:12 PMFelix Ruess
07/07/2023, 1:06 PMdynamic
decorator as well...Geert
07/07/2023, 6:28 PMn1-0-dn3
etc.? That would be nice to quickly see on first glance in the UI on what item a failure occured (ie. did it fail on i=1
, i=2
, ..?), without having to go into the InputsThomas Blom
07/07/2023, 9:38 PMa(i=i).with_overrides(node_name=f"a: i={i}")
This is demonstrated as above in docs about subworkflows (which I realize is different than the above use-case, but may still work for you).Geert
07/10/2023, 10:13 AM