Hi, does anyone know where this error could come from?

[49]: code:"ContainersNotReady|CreateContainerError" message:"Grace period [3m0s] exceeded|containers with unready status: [amj9ml8vx26pqmz5dmgg-n10-0-n2-0-37]|context deadline exceeded"

Interestingly, the mentioned pod shows up as completed in the kubectl output

Sometimes I am also getting same issue

@white-teacher-47376 could you share a

kubectl describe

of the Pod? and also

kubectl logs ...

?

Thanks David, there was this error in the kubectl describe output, I missed that one before: Error: failed to reserve container name. There is already a thread for that issue: https://flyte-org.slack.com/archives/CP2HDHKE1/p1720775616159299 I was able to solve the problem with the sugessted solution:

create-container-error-grace-period: 10m

FYI: We are using bare-metal kubernetes, not GKE - but it probably depends on our kubelet config.