Hi team.. I’m suddenly running into a weird error ...
# ask-the-community
Hi team.. I’m suddenly running into a weird error with a map task (3rd time now). Some of the map tasks fail with this error and eventually the whole workflow fails. Nothing has changed as far as our config or workflow code is concerned. The task has
Copy code
code:"UnexpectedObjectDeletion" message:"object [<k8s-namespace-name>/a5vmlnn7xxvpbjv5qztd-n6-0-33] terminated in the background, manually
Also are successful map tasks not cached? There were 20 that succeeded out of 50 but looks like on a recover/relaunch they are running all over again
Turn on inject-finalizer - you are running a lot of pods and they are being deleted by k8s
Thanks.. let me try this. It’s curious since this workflow has been running for several months now\
Looks like we already have it set to true for copilot
Do the propeller and scheduler also need it?
ohh you need it for propeller
what is for co-pilot
that is not needed
This is a screenshot of the yaml from flyte example setup… So should the k8s plugin config with inject finalizer be under core -> propeller -> plugins -> k8s