Is there a way to do multiple map tasks in a row w...
# flyte-supportf
freezing-tailor-85994
04/25/2025, 6:03 PMIs there a way to do multiple map tasks in a row without doing a coalesce/resplit in the middle. My thought is to construct a dataset, chunk it up, build data modules out of each chunk (map task 1) and then run ML inference on each data module (task 2)
a
average-finland-92144
04/29/2025, 7:05 PMAFAICT you should be able to chain map_task 1 to map_task 2
average-finland-92144
04/29/2025, 7:07 PMdoes that help?
Also I guess your inference use case has some room for latency? Flyte MapTasks, in their current version, still suffer from container boot up penalty
f
freezing-tailor-85994
04/29/2025, 7:08 PMyep, thanks!
2 Views