<#7362 fix(flytek8s): classify ImagePullBackOff ca...
# flyte-github
a
#7362 fix(flytek8s): classify ImagePullBackOff caused by registry 429 as system error Pull request opened by pingsutw Tracking issue N/A Why are the changes needed? Container runtimes surface registry rate limiting (HTTP 429 Too Many Requests) through the
ImagePullBackOff
waiting reason. Today, once the
ImagePullBackoffGracePeriod
elapses,
DemystifyPending
returns
PhaseInfoRetryableFailureWithCleanup
, which is classified as a USER error and consumes the user's retry budget. 429s from the registry are an infrastructure problem outside the user's control — typically transient throttling on the registry side. Misclassifying them as user errors means a transient registry rate-limit can exhaust a task's user retries even though the user did nothing wrong. Example real-world pod waiting message:
Copy code
Back-off pulling image "us-central1-docker.pkg.dev/.../internal-gnina:...": ErrImagePull: failed to pull and unpack image "...": failed to copy: httpReadSeeker: failed open: unexpected status code https://us-central1-docker.pkg.dev/v2/...: 429 Too Many Requests
What changes were proposed in this pull request? • `pod_helper.go`: in
classifyWaitingContainer
, after the
ImagePullBackOff
grace period, detect the
429 Too Many Requests
substring in the waiting message and return
PhaseInfoSystemRetryableFailureWithCleanup
instead of
PhaseInfoRetryableFailureWithCleanup
. • Add small helper
isRegistryRateLimited(message string) bool
. • All other
ImagePullBackOff
messages keep their existing user-error classification. How was this patch tested? • New unit test
TestDemystifyPending/ImagePullBackOffOutsideGracePeriod_RegistryRateLimited
exercises a realistic 429 waiting message and asserts
core.ExecutionError_SYSTEM
. • Existing
ImagePullBackOffOutsideGracePeriod
(non-429) test continues to pass, confirming the default classification is unchanged. •
go test ./go/tasks/pluginmachinery/flytek8s/...
passes locally. Labels • fixed Setup process N/A Screenshots N/A Check all the applicable boxes • I updated the documentation accordingly. • All new and existing tests passed. • All commits are signed-off. Related PRs N/A Docs link N/A flyteorg/flyte