We've recently upgraded to flyte

1.10.7

and seem to have a potential bug happening when we try to do

pyflyte run --remote --copy-all ...

.Both the sever and flytekit are on

1.10.7

version. We use PKCE auth w/ IdP, and during the scheduling of the execution, we get this error:

E0307 21:26:17.332123       1 workers.go:103] error syncing 'flyte-test-development/f95b4da790bce43a99a4': failed at Node[n0]. RuntimeExecutionError: failed during plugin execution, caused by: failed to execute handle for plugin [container]: [Invalid] failed to create resource, caused by: Pod "f95b4da790bce43a99a4-n0-0" is invalid: metadata.labels: Invalid value: "<mailto:my.email@gmail.com|my.email@gmail.com>": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue',  or 'my_value',  or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')
{"json":{"src":"controller.go:152"},"level":"info","msg":"==> Enqueueing workflow [flyte-test-development/f95b4da790bce43a99a4]","ts":"2024-03-07T21:26:17Z"}

Is it trying to set my username as a label somewhere on the pod, without specifying the key maybe? Has anyone else ran into this?

I do not see this bug on our deployments. but cc @cool-lifeguard-49380 / @high-accountant-32689 fyi?

My guess is that if the IdP uses email as the sub(which Okta does) that this happens

Hey @alert-oil-1341, you are right, this is related to the PR you linked. We didn’t run into this because our idp uses a numerical identifier as exec identity instead of an email, sorry about this. I fix this by sanitizing the exec id before injecting it as a label in this PR: https://github.com/flyteorg/flyte/pull/5023