Hello again! Is there any guidance on how to handl...
# flyte-v2-supportb
brash-nest-94989
08/21/2026, 4:34 PMHello again! Is there any guidance on how to handle workload disruptions with node pool updates when using flyte 2 binary? I did a node pool update today that rerolled the node flyte 2 was running on and a couple of workloads got aborted as a result
f
freezing-airport-6809
08/21/2026, 11:35 PM
really can you share how long was the update for
freezing-airport-6809
08/21/2026, 11:36 PM
this is not expected, also the flyte-core is in progress that will provide more replicas to you can do a rolling deployment
🙏 1
🦜 1
freezing-airport-6809
08/21/2026, 11:36 PM
cc @glamorous-carpet-83516 this should not happen right, controller should keep retrying
g
glamorous-carpet-83516
08/21/2026, 11:43 PMit should not happen. do you remember when did it get aborted. is it immediately after you restart flyte pod?
b
brash-nest-94989
08/25/2026, 1:46 PMYea - it happened after flyte pod was restarted, I think the outage was probably somewhere around 1-3 mins from node spin down to a new one spinning up
g
glamorous-carpet-83516
08/25/2026, 5:21 PM@faint-pilot-24699 mind taking a look at this
f
faint-pilot-24699
08/26/2026, 12:38 AMWill take a look
faint-pilot-24699
08/26/2026, 4:08 AM@brash-nest-94989 I tried to reproduce this. Flyte will recreate the pod when a pod is evicted. However, it will abort an action when the CR is deleted(CR's DeletionTimestamp is marked), and I'm thinking that might be the reason for your issue. Would you able to check whether that happened when you reroll your node pool?
2 Views