acoustic-carpenter-78188
04/27/2026, 6:41 AMflyte-pod-webhook denies the request because a referenced secret cannot be found in any configured secret manager), launchResource in executor/pkg/plugin/k8s/plugin_manager.go returned the error as a raw Go error with pluginsCore.UnknownTransition and no phase transition.
In taskaction_controller.go, when Plugin.Handle returns a Go error, the controller only logs "Plugin Handle failed", emits a FailedPluginHandle event, and requeues with the default duration. It does not record a phase, increment any system-failure counter, or convert to PermanentFailure. As a result, the TaskAction stays in Queued forever — the webhook will keep denying the same request, and there is no max-system-failure handling on this path.
Repro:
@env.task
def fn(x: int) -> int:
print("my secret is: ", os.getenv("TEST1")) # secret "test1" does not exist
...
Before the fix, the TaskAction loops forever with the webhook denial message in the controller log:
admission webhook "flyte-pod-webhook.flyte.org" denied the request:
none of the secret managers injected secret [key:"test1" ...]
What changes were proposed in this pull request?
In launchResource, before the generic system-error fall-through:
1. Detect admission-webhook denials via a small helper isAdmissionWebhookDenial(err) that matches the canonical error string ("admission webhook" + "denied the request").
2. Also detect k8serrors.IsInvalid(err) (invalid pod specs are equally non-transient).
3. Return pluginsCore.PhaseInfoFailure("AdmissionDenied", err.Error(), nil) — a permanent failure phase — so the TaskAction terminates immediately with the underlying webhook message preserved for the user.
How was this patch tested?
Screenshot 2026-04-27 at 11 56 45 AM
Manual end-to-end test using the bundled devbox:
1. Rebuilt flyte-devbox:latest with this change (make -C docker/devbox-bundled build) and restarted the devbox.
2. Ran a task that references a non-existent secret (secrets=["test1"]).
3. Confirmed the TaskAction transitions to PermanentFailure in ~8s (single attempt) instead of getting stuck:
NAMESPACE NAME RUN TASKTYPE STATUS AGE
flytesnacks-development rf5v6h9d79n2ndnnqkpr-a0 rf5v6h9d79n2ndnnqkpr python PermanentFailure 8s
`Error State`:
Code: AdmissionDenied
Kind: USER
Message: admission webhook "flyte-pod-webhook.flyte.org" denied the request: ...
Labels
• fixed
Check all the applicable boxes
• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.
• main
• #6583
• fix(executor): bound consecutive Plugin.Handle errors with a system-failure counter 👈
flyteorg/flyteacoustic-carpenter-78188
04/27/2026, 8:02 PM