https://flyte.org logo
#ask-the-community
Title
# ask-the-community
g

Geert

07/07/2023, 11:55 AM
Is there a way to allow tasks to fail? I have the following setup:
Copy code
@dynamic
def base_workflow(config: Config):
    for i in [1..10]:
        task_a = a(i=i)
        task_b = b(i=i)
        task_c = c(i=i)
        task_a >> task_b >> task_c
This all starts nicely 10x in parallel (as desired), but as soon as one of the tasks fail (lets say, task_a fails for i=3), they all abort. Is there a Flyte-native way to allow i=3 to fail, but let the others complete?
m

Maarten de Jong

07/07/2023, 12:03 PM
For a workflow you can set a failure policy, to allow remaining tasks to continue.
@workflow(failure_policy=WorkflowFailurePolicy.FAIL_AFTER_EXECUTABLE_NODES_COMPLETE)
g

Geert

07/07/2023, 12:08 PM
That sounds great, unfortunately
@dynamic
doesn’t have that field, but maybe I can find a way to nest a workflow or something to use it.
m

Maarten de Jong

07/07/2023, 12:09 PM
Setting it on the top level should propagate it downward, so if you wrap your dynamic in a workflow that should work out
g

Geert

07/07/2023, 12:12 PM
Works! Thanks a lot @Maarten de Jong 🙌
f

Felix Ruess

07/07/2023, 1:06 PM
we also encountered this, would be nice to add that field to the
dynamic
decorator as well...
g

Geert

07/07/2023, 6:28 PM
Maybe you know this as well @Maarten de Jong @Felix Ruess since you're also working with dynamic tasks/workflows: Is there a way to add a prefix/label/name to the tasks, instead of the rather cryptic
n1-0-dn3
etc.? That would be nice to quickly see on first glance in the UI on what item a failure occured (ie. did it fail on
i=1
,
i=2
, ..?), without having to go into the Inputs
t

Thomas Blom

07/07/2023, 9:38 PM
@Geert I have not investigated this, but you may be able to use something like
Copy code
a(i=i).with_overrides(node_name=f"a: i={i}")
This is demonstrated as above in docs about subworkflows (which I realize is different than the above use-case, but may still work for you).
g

Geert

07/10/2023, 10:13 AM
Thanks @Thomas Blom! Interesting, it seems it picks up the names when the task did not run yet. When they are running/have completed, they switch back to a random identifier:
For now this at least gives us some (temporary) visibility, much appreciated 🙏
35 Views