millions-judge-87871
11/03/2023, 3:11 PMmap_task
and partial
with different list
objects. If there is a better way to do this behaviour please let me know.
When using partial
to prepare a function for a map_task
if you partially fill in a list
object the map task requires that the list you are mapping over is the same length as the list you are partially filling with the following error: all maptask input lists must be the same length
. I don't think this should be expected behaviour as I am not running the map_task
over both lists. An example workflow is below:
from functools import partial
from flytekit import map_task, task, workflow
@task
def add(a_list: list[int], b: int) -> list[int]:
return [a + b for a in a_list]
@workflow
def adding_many_wf(a_list: list[int] = [1, 2, 1], b_list: list[int] = [1, 2, 3]) -> list[list[int]]:
part_add = partial(add, a_list=a_list)
output = map_task(part_add)(b=b_list)
return output
So long as a_list
and b_list
in the above workflow are the same length it will run. If b_list
is [1, 2]
then it will error with the above.
Any help would be greatly appreciated. Thanks.tall-lock-23197
millions-judge-87871
11/06/2023, 9:13 AM