<#878 [Core Feature] Allow Some Dynamic Sub-workfl...
# flyte-github
a
#878 [Core Feature] Allow Some Dynamic Sub-workflows to Fail Issue created by michaels-lyft Motivation: Why do you think this is important? We have a lot of scenarios where there's a dynamic_task that executes a list of sub-workflows or sub-dynamic_tasks. Today, when one of these sub-executions fail, it would cause the parent dynamic_task to fail even with
FAIL_AFTER_EXECUTABLE_NODES_COMPLETE
enabled on the parent workflow or
allowed_failure_ratio
enabled on the dynamic_task (which only works with sub-python_tasks). This aborts any of the dependent downstream tasks automatically. But sometimes, we would like to allow certain % of the tasks to fail since we may not always need 100% of the output. Goal: What should the final outcome look like, ideally? There should be an option to allow a % of the sub-workflows or sub-dynamic_tasks to fail. Describe alternatives you've considered Gone through the discussions at #191. Tried using
FAIL_AFTER_EXECUTABLE_NODES_COMPLETE
and
allowed_failure_ratio
but neither works in this scenario. [Optional] Propose: Link/Inline OR Additional context I can think of two ways to do this: 1. Expand
allowed_failure_ratio
to include sub-workflows and sub-dynamic_tasks 2. Add something similar to a
try-catch
in the dynamic_task that we could use to catch any sub-execution failures and make the parent task succeed (as suggested here). flyteorg/flyte