acoustic-carpenter-78188
05/22/2026, 7:09 PMReplicatedJobsStatus[workers].Failed > 0, calls flytek8s.DemystifyFailedOrPendingPod on the rank-0 pod (<jobset>-workers-0-0) to surface failures before the JobSet controller sets JobSetFailed
• Adds `maybeSystemRetryOnMaintenance`: on JobSetFailed, if RestartOnHostMaintenance=true, inspects the rank-0 pod and reclassifies system-caused evictions as PhaseInfoSystemRetryableFailureWithCleanup — retrying without charging the user's max_restarts budget
• GetTaskPhase now reads ClusteredTaskSpec to check FailurePolicy.RestartOnHostMaintenance flag; both detection paths are best-effort (fall through to normal failure if the pod is already cleaned up)
How was this patch tested?
• Run: go test ./flyteplugins/go/tasks/plugins/k8s/clustered/... -v — all 15 tests pass
• `TestGetTaskPhase_FastFail_NoJobsFailed`: confirms no pod lookup when ReplicatedJobsStatus[workers].Failed == 0 → returns PhaseRunning
• `TestGetTaskPhase_MaintenanceRetry_FlagFalse`: confirms JobSetFailed returns PhaseRetryableFailure when RestartOnHostMaintenance=false, no pod lookup attempted
• Integration coverage (actual eviction detection via DemystifyFailedOrPendingPod) is deferred to a later PR covering E2E tests
Setup process
Screenshots
Check all the applicable boxes
• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.
Related PRs
Stack
If you do use git town to manage PR Stacks, the stack relevant to this PR
will show below. Otherwise, you can ignore this section.
• main
• #7417
• feat: add worker-0 fast-fail and host maintenance retry to clustered plugin 👈
Docs link
flyteorg/flyteacoustic-carpenter-78188
05/27/2026, 9:39 PM