<#7153 Use retryable failure for failed Ray job de...
# flyte-github
a
#7153 Use retryable failure for failed Ray job deployments Pull request opened by pingsutw Tracking issue Why are the changes needed? When a Ray job deployment fails, the plugin should return a retryable failure (system error) instead of a permanent failure. This allows Flyte to retry the task, which is the desired behavior since deployment failures are often transient (e.g., cluster scheduling issues, temporary resource constraints). What changes were proposed in this pull request? • Changed
JobDeploymentStatusFailed
handler from
PhaseInfoFailure
to
PhaseInfoSystemRetryableFailureWithCleanup
so failed Ray jobs are retried • Added
TestBuildResourceRayMissingPrimaryContainer
- validates error when primary container is missing from pod spec • Added
TestGetTaskPhaseFailedRetryable
- validates retryable failure phase with reason/message propagation • Updated
TestGetTaskPhase
to expect
PhaseRetryableFailure
for failed deployments How was this patch tested? Unit tests added and all existing tests pass:
Copy code
go test ./go/tasks/plugins/k8s/ray/ -count=1
Labels • changed: Ray job failure handling changed to retryable • added: New unit tests Check all the applicable boxes • I updated the documentation accordingly. • All new and existing tests passed. • All commits are signed-off. flyteorg/flyte