Hey folks, trying to debug an issue. We’re seeing various tasks fail with
terminated in the background, manually
. I took a look at the code here and see that this occurs when the pod get deleted. I don’t see a delete request being sent to admin so don’t think there is anything there. Propeller logs haven’t shown anything either. So I’ve been looking at kube logs to try to see what is triggering the pod deleted but there’s a lot of noise in the logs so taking some time to through them. Meanwhile, I figured I’d ask here if anyone has run into something similar before
f
freezing-airport-6809
03/30/2022, 8:49 PM
do you have inject-finalizer enabled?
freezing-airport-6809
03/30/2022, 8:49 PM
if not can you set it to true?
freezing-airport-6809
03/30/2022, 8:49 PM
this would prevent such random deletions happening
freezing-airport-6809
03/30/2022, 8:49 PM
so it might have completed and Kube GC deleted it. When there are too many pods (pod pressure), this happens
g
glamorous-father-92994
03/30/2022, 9:08 PM
Let me check if we have that enabled
glamorous-father-92994
03/30/2022, 9:11 PM
Looks like we do not have it enabled. I'll enable it