Hi everyone, Is it possible to run part of a worklow or a list of tasks in a single pod? I would like to use the caching mechanism for tasks but reduce the cost of starting a pod for each task. I have a task that i want to call up thousands of times, but it doesn't require a lot of computing time

hi @Tim Hebbeler have you checked Map Tasks? Your use case sounds like a fit https://docs.flyte.org/projects/cookbook/en/latest/auto/core/control_flow/map_task.html

But from what i read in the K8s Array plugin a pod is started for each instance in the array.

you want to reuse the same pod to run the other tasks, right?Flyte doesn’t support it yet.

yes, exactly

cc @Ketan (kumare3)

thank you both very much for the replies. Do you think it is enough to just adapt the k8s array plugin or are extensive changes necessary?

No so we have been thinking of this