Does anyone know if map tasks (array) + dynamic (+...
# flyte-support
c
Does anyone know if map tasks (array) + dynamic (+
min_success_ratio
) is expected to work? Details in 🧵
Copy code
from typing import Optional
from flytekit import task, workflow, map_task, dynamic


@task
def foo_task(inp: str) -> float:
    # Might or might not fail
    return 0.5

@dynamic
def foo_dynamic(inp: str) -> float:
    # Multiple tasks need to be called here, dynamic is in fact needed
    res = foo_task(inp=inp)
    return res

@task
def report(res: list[Optional[float]]):
    # Process and report all successful results
    print(f"Results: {res}")

@workflow
def wf():
    # Run dynamic subworkflow for multiple inputs, we expect that some might fail
    res = map_task(foo_dynamic, min_success_ratio=0.25)(inp=["foo", "bar"])
    # Process the results of the succesful dyn subworkflows
    report(res=res)

if __name__ == "__main__":
    wf()
When running locally with python, I get this error:
Copy code
TypeError: Failed to convert outputs of task
'wf.map_foo_dynamic_13e5678cbbca248d78d96e312e72de26-arraynode' at position 0.
Failed to convert type <class 'list'> to type typing.List[typing.Optional[float]].
Error Message: Cannot convert from [Flyte Serialized object: Type: <LiteralMap> Value: <literals {
key: "o0" value { scalar { primitive { float_value: 0.5 } } } }>] to typing.Optional[float].
I can
pyflyte run
the workflow in the cluster but then get the following error in the dynamic task:
Copy code
File "/usr/local/lib/python3.10/site-packages/flytekit/tools/translator.py", line 498, in get_serializable_node
        task_spec = get_serializable(entity_mapping, settings, entity.flyte_entity, options=options)
      File "/usr/local/lib/python3.10/site-packages/flytekit/tools/translator.py", line 789, in get_serializable
        cp_entity = get_serializable_task(entity_mapping, settings, entity)
      File "/usr/local/lib/python3.10/site-packages/flytekit/tools/translator.py", line 176, in get_serializable_task
        settings.project,

Message:

    AttributeError: 'NoneType' object has no attribute 'project'

User error.
The respective line in translator hasn’t been changed for years which is why I’m surprised why this suddenly appears here 🤔
So my question: do we expect dynamic to work in a map task in the first place?
t
@cool-lifeguard-49380 this isn’t supported sorry
flytekit should probably flag this.
mapping over things other than a normal task is something we’ve considered in the past though. do you want to create an issue for this?
let’s see how useful this will be to the community
c
Ok, thanks for the clarification! I’ll create issues both for flagging and for the feature.
> flytekit should probably flag this. https://github.com/flyteorg/flytekit/pull/2699 <- @thankful-minister-83577 added a check, could you please review?
And I created an issue with the user story: https://github.com/flyteorg/flyte/issues/5684
🙏