<#7183 Fix copilot TerminationGracePeriodSeconds s...
# flyte-github
a
#7183 Fix copilot TerminationGracePeriodSeconds set to nanoseconds instead of seconds Pull request opened by bergman Why are the changes needed?
AddCoPilotToPod
sets
TerminationGracePeriodSeconds
by casting a
time.Duration
(which is stored as nanoseconds) directly to `*int64`: coPilotPod.TerminationGracePeriodSeconds = (*int64)(&cfg.Timeout.Duration)
TerminationGracePeriodSeconds
expects seconds, not nanoseconds. With the default 1-hour timeout, this results in a grace period of ~114,000 years (
3,600,000,000,000
nanoseconds interpreted as seconds) instead of the intended 3,600 seconds. This causes pods with copilot sidecars to stay in
Terminating
state effectively forever when a node is drained, since Kubernetes waits for the grace period to expire before sending SIGKILL. We discovered this while rotating GKE node pools in production. Pods with copilot sidecars were stuck in
Terminating
state indefinitely, blocking node drain. Kubernetes was honoring the ~114,000-year grace period and never sending SIGKILL, making it impossible to cleanly decommission nodes without force-deleting pods. What changes were proposed in this pull request? Replace the unsafe pointer cast with a proper conversion using `Duration.Seconds()`: coPilotPod.TerminationGracePeriodSeconds = ptr.To(int64(cfg.Timeout.Duration.Seconds())) This also eliminates the pointer aliasing into the config struct that the original code introduced. How was this patch tested? Added a unit test
TestAddCoPilotToPod/grace-period
that sets a 1-hour timeout and asserts that
TerminationGracePeriodSeconds
is
3600
(seconds), not
3600000000000
(nanoseconds). Labels • fixed Check all the applicable boxes • I updated the documentation accordingly. • All new and existing tests passed. • All commits are signed-off. Related PRs #6501 — the PR that introduced the bug flyteorg/flyte