https://flyte.org logo
#ask-the-community
Title
# ask-the-community
f

Frank Shen

04/11/2023, 5:11 PM
Hello, I have a workflow that executes training tasks parallel. If one of the parallel training tasks failed, I observed that all the other tasks are aborted by Flyte. How do I change the config so that other parallel tasks and their downstream tasks will continue to the end, even when one or two training tasks failed?
Copy code
@workflow
def wf_train_parallel():
    for tenure in range(1, 25, 1):
        xgb_train(tenure=tenure)
        downstream_task()
...

@task
def xgb_train(tenure: int)
...
@task
def downstream_task()
...
g

Greg Gydush

04/11/2023, 6:06 PM
wf_train_parallel
should be a dynamic, right? You can set the failure policy:
Copy code
from flytekit import workflow
from flytekit.core.workflow import WorkflowFailurePolicy

@workflow(failure_policy=WorkflowFailurePolicy.FAIL_AFTER_EXECUTABLE_NODES_COMPLETE)
def test_workflow(...) -> None:
   ...
7 Views