acoustic-carpenter-78188
07/25/2023, 1:41 PMmap_task
with partial
and fixed List
inputs, the error message
ValueError: Map tasks do not support partial tasks with lists as inputs.
does not appear when the List
input is the output of a previous task.
Flyte does not recognise that the output of a task is a List
here.
This is linked to #3884.
Expected behavior
This error message should always appear when a List
is input, otherwise it can be very difficult to debug. In some cases, the function will run with no error but the output will be wrong.
Additional context to reproduce
@task
def get_x() -> List[float]:
return [1.0, 2.4, 3.9]
@task
def t1(x: List[float], y: float) -> List[float]:
return x + [y]
@workflow
def wf(y: List[float]) -> List[List[float]]:
x = get_x()
partial_t1 = partial(t1, x=x)
return map_task(partial_t1)(y=y)
When running wf(y=[0.1, 0.3, 0.5, 0.7, 0.9])
, t1
will be mapped according to x=[1.0, 2.4, 3.9]
.
Three map tasks will run, one for each value in x
.
The output will be
[1.0, 2.4, 3.9, 0.3],
[1.0, 2.4, 3.9, 0.5]]
instead of
[1.0, 2.4, 3.9, 0.3],
[1.0, 2.4, 3.9, 0.5],
[1.0, 2.4, 3.9, 0.7],
[1.0, 2.4, 3.9, 0.9]]
No error message will be thrown.
Screenshots
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyte