acoustic-carpenter-78188
09/02/2023, 12:38 AMFAIL_AFTER_EXECUTABLE_NODES_COMPLETE enabled on the parent workflow or allowed_failure_ratio enabled on the dynamic_task (which only works with sub-python_tasks). This aborts any of the dependent downstream tasks automatically. But sometimes, we would like to allow certain % of the tasks to fail since we may not always need 100% of the output.
Goal: What should the final outcome look like, ideally?
There should be an option to allow a % of the sub-workflows or sub-dynamic_tasks to fail.
Describe alternatives you've considered
Gone through the discussions at #191. Tried using FAIL_AFTER_EXECUTABLE_NODES_COMPLETE and allowed_failure_ratio but neither works in this scenario.
[Optional] Propose: Link/Inline OR Additional context
I can think of two ways to do this:
1. Expand allowed_failure_ratio to include sub-workflows and sub-dynamic_tasks
2. Add something similar to a try-catch in the dynamic_task that we could use to catch any sub-execution failures and make the parent task succeed (as suggested here).
flyteorg/flyteacoustic-carpenter-78188
09/02/2023, 12:38 AM