acoustic-carpenter-78188
07/25/2023, 1:41 PMmap_task with partial and fixed List inputs, the error message
ValueError: Map tasks do not support partial tasks with lists as inputs.
does not appear when the List input is the output of a previous task.
Flyte does not recognise that the output of a task is a List here.
This is linked to #3884.
Expected behavior
This error message should always appear when a List is input, otherwise it can be very difficult to debug. In some cases, the function will run with no error but the output will be wrong.
Additional context to reproduce
@task
def get_x() -> List[float]:
return [1.0, 2.4, 3.9]
@task
def t1(x: List[float], y: float) -> List[float]:
return x + [y]
@workflow
def wf(y: List[float]) -> List[List[float]]:
x = get_x()
partial_t1 = partial(t1, x=x)
return map_task(partial_t1)(y=y)
When running wf(y=[0.1, 0.3, 0.5, 0.7, 0.9]), t1 will be mapped according to x=[1.0, 2.4, 3.9].
Three map tasks will run, one for each value in x.
The output will be
[1.0, 2.4, 3.9, 0.3],
[1.0, 2.4, 3.9, 0.5]]
instead of
[1.0, 2.4, 3.9, 0.3],
[1.0, 2.4, 3.9, 0.5],
[1.0, 2.4, 3.9, 0.7],
[1.0, 2.4, 3.9, 0.9]]
No error message will be thrown.
Screenshots
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyte