orange-hairdresser-63684
04/18/2022, 6:30 PMterminated in the background, manually
intermittently. No errors are reported in the POD log. Any suggestions on how to troubleshoot?high-accountant-32689
04/18/2022, 6:35 PMhallowed-mouse-14616
04/18/2022, 6:38 PMhallowed-mouse-14616
04/18/2022, 6:38 PMhallowed-mouse-14616
04/18/2022, 6:39 PMorange-hairdresser-63684
04/18/2022, 6:43 PMorange-hairdresser-63684
04/18/2022, 6:45 PMterminated in the background, manually
and no POD logs at all)hallowed-mouse-14616
04/18/2022, 6:49 PMkubectl describe pod
/ kubectl logs ...
) show nothing interesting?orange-hairdresser-63684
04/18/2022, 6:53 PMterminated in the background, manually
. We send logs to stackdriver and nothing interesting there. I can't do describe/logs because by the time we learn about the issue the pod is gonehallowed-mouse-14616
04/18/2022, 6:57 PMhallowed-mouse-14616
04/18/2022, 6:58 PM--plugins.k8s.delete-resource-on-finalize Instructs the system to delete the resource on finalize. This ensures that no resources are kept around (potentially consuming cluster resources). This, however, will cause k8s log links to expire as soon as the resource is finalized.
--plugins.k8s.inject-finalizer Instructs the plugin to inject a finalizer on startTask and remove it on task termination.
hallowed-mouse-14616
04/18/2022, 6:58 PMhallowed-mouse-14616
04/18/2022, 6:59 PMorange-hairdresser-63684
04/18/2022, 7:00 PMinject-finalizer
enabledorange-hairdresser-63684
04/18/2022, 7:02 PMinject-finalizer
enabled, are you saying the pods should not be deleted?freezing-airport-6809
freezing-airport-6809
freezing-airport-6809
freezing-airport-6809
orange-hairdresser-63684
04/18/2022, 7:04 PMdisabling the delete resource on finalize means that propeller will not delete the pod when it find a terminal state.but as you said earlier
This means that something that is not Flyte deletes the pod.
orange-hairdresser-63684
04/18/2022, 7:04 PMfreezing-airport-6809
freezing-airport-6809
freezing-airport-6809
hallowed-mouse-14616
04/18/2022, 8:57 PMorange-hairdresser-63684
04/22/2022, 6:01 PM