Hi all, I have a

ContainerTask

with custom

PodTemplate

, and it seems like Flyte is unable to detect when the task/container has finished and written the output file. I also have a different

ContainerTask

that doesn't use

PodTemplate

and everything works fine there. While looking at the task tab under execution for both, I see that the working task has

container.dataConfig

while the other task has

k8sPod.dataConfig

, but when I go to the task definition itself I don't see any

dataConfig

anywhere, for the task that isn't working. What am I doing wrong? Maybe I need to specify the

dataConfig

manually inside the

PodTemplate

? although I don't see where

Hey @Lotem Yehonatan Amiel, can you share your code that produces either? pod.DataConfig need to be populated yes! After a quick search I think this is a bug.. do you mind filing here: github.com/flyteorg/flyte/issues ?

Hi @Haytham Abuelfutuh, thanks for the reply! It's a bit long so I put it all into 1 file I will open an issue if you still think it's a bug after you've looked at the code But I would also like to hear if you have any idea for a temporary workaround, as I need this to work very soon-ish 😄

Sorry for the late response... One more thing to debug, can you check the Pod Spec that was generated when you ran that task?

@Haytham Abuelfutuh also sorry for late response, didn't see you commented. I'll try to get a pod spec for you soon, but I did see that the copilot container wasn't added in these cases where i was using pod spec i'm 99% sure the issue is with the backend (propeller i think? not sure), which either doesnt check the

k8sPod.dataConfig

or just decides not to create a

copilot

for those my workaround for now is using "Runtime default pod spec", so my tasks have

container.dataConfig

the problem is sometimes i need a different pod spec (to add another sidecar), so this workaround isnt great