Hi everyone Is it possible to run part of a worklow or a lis

Question

Hi everyone Is it possible to run part of a worklow or a list of tasks in a single pod I would like to use the caching mechanism for tasks but reduce the cost of starting a pod for each task I have a task that i want to call up thousands of times but it doesn t require a lot of computing time

David Espejo (he/him) · Answer

hi < Tim Hebbeler> have you checked Map Tasks Your use case sounds like a fit

Tim Hebbeler · Answer

But from what i read in the K8s Array plugin a pod is started for each instance in the array

Kevin Su · Answer

you want to reuse the same pod to run the other tasks right Flyte doesn t support it yet

Tim Hebbeler · Answer

yes exactly

Kevin Su · Answer

cc < Ketan kumare3 >

Tim Hebbeler · Answer

thank you both very much for the replies Do you think it is enough to just adapt the k8s array plugin or are extensive changes necessary

Ketan (kumare3) · Answer

No so we have been thinking of this

Ketan (kumare3) · Answer

< Dan Rammer hamersaw > is almost done completely reengineering array map tasks this will give us the capability of reusing pods

Ketan (kumare3) · Answer

We would Love to work with you if you can contribute

Ketan (kumare3) · Answer

Ohh you want to call a 1000 times I also think this is a good case for flytekit plugins

Ketan (kumare3) · Answer

Backend flytekit plugin I mean