Harshit Sharma09/06/2022, 12:03 AM
Yuvraj09/06/2022, 8:48 AM
Harshit Sharma09/06/2022, 10:09 AM
Yuvraj09/06/2022, 11:03 AM
Harshit Sharma09/06/2022, 11:04 AM
from flytekit import Resources, task @task(requests=Resources(cpu="2", mem="1000Mi")) def fetch_domain_list_df(...): ...
Harshit Sharma09/06/2022, 11:34 AM
Harshit Sharma09/06/2022, 12:46 PM
Harshit Sharma09/07/2022, 11:17 AM
Dan Rammer (hamersaw)09/07/2022, 2:01 PM
configuration option which uses k8s finalizers to stop external systems from deleting the Pods, but some external systems do not respect this scheme and delete anyway. So this is a "best effort" by Flyte to keep Pods around, if they are still being deleted then unforunately there is not much we can do to stop it. A few ideas here are to look at the k8s logs to determine what is deleting the pods, additionally you can reduce the concurrency configuration on the map task so fewer Pods are spawned (typically these are deleted by some sort of resource manager). (3) The issue above (2788) has a submitted fix - but it resulted in a panic on externally deleted pods during interruptible failures. Are you designating the task as "interruptible"?
Harshit Sharma09/07/2022, 3:29 PM
Dan Rammer (hamersaw)09/07/2022, 4:23 PM
Harshit Sharma09/07/2022, 4:32 PM
Harshit Sharma09/08/2022, 9:31 AM