Nicholas LoFaso
01/31/2022, 6:00 PMtsk
tasks as part of a dynamic workflow and all 10 are deleted from k8s as soon as they completeYee
Dan Rammer (hamersaw)
02/01/2022, 1:21 PMNicholas LoFaso
02/01/2022, 1:23 PMmap_tasks
, but I noticed the same behavior with map_tasks
as the @dynamic
tasks as well. If that helps troubleshooting.Dan Rammer (hamersaw)
02/03/2022, 3:59 PMdelete-resource-on-finalize
that is set on the plugins configmap for flytepropeller. It controls, as you may guess, whether or not k8s resources are deleted when flyte attempts to finalize the task. In your above configuration this may look like:
configmap:
k8s:
plugins:
k8s:
default-env-vars:
- FLYTE_STATSD_HOST: "flytestatsd.datadog.svc.cluster.local"
- FLYTE_STATSD_DISABLE_TAGS: "True"
delete-resource-on-finalize: false
create-container-error-grace-period: 8m0s
In my testing I was able to not delete dynamic task pods on completion / failure. Let me know if this works for you and / or if you run into any more issues.Nicholas LoFaso
02/04/2022, 2:43 PMdelete-resource-on-finalize: false
and hopefully that will resolve the issue