<#5684 [Core feature] Map node array tasks do not ...
# flyte-github
a
#5684 [Core feature] Map node array tasks do not support `@dynamic`, `@eager`, or `@workflow` nodes Issue created by fg91 ### Motivation: Why do you think this is important? Map node array tasks allow users to map a
@task
over a list of inputs. Currently, other node types, i.e.
@dynamic
,
@eager
, and
@workflow
are not supported. As a user, I would like to implement the following logic that today cannot be expressed in Flyte due to this limitation: @workflow def map_workflow_consume_partial_results(data: list[int] = [10, 12, 11, 10, 13, 12, 100, 11, 12, 10]): partial_results = map_task(dynamic_subwf, min_success_ratio=0.75)(data_point=data) consume_partial_results_task(results=partial_results) In this example, the task
consume_partial_results_task
can consume a subset of the outputs of
dynamic_subwf
(or a normal sub workflow) which is being mapped even if not all (dynamic) sub workflows are successful. Currently, consuming partial results can only be done when mapping a
@task
but not all logic can be compressed into a single task. ### Goal: What should the final outcome look like, ideally? Support every node type in
map_task
. ### Describe alternatives you've considered One can execute multiple (dynamic) sub workflows and tolerate failures of some of them by using
WorkflowFailurePolicy.FAIL_AFTER_EXECUTABLE_NODES_COMPLETE
but this does not allow consuming partial results of the successful (dynamic) sub workflows. ### Propose: Link/Inline OR Additional context No response ### Are you sure this issue hasn't been raised already? • Yes ### Have you read the Code of Conduct? • Yes flyteorg/flyte