I'm trying to configure auth on my flyte-binary helm build. I can authenticate with the UI, flytectl, and pyflyte. But all of the workflow executions show a status of "UNKNOWN" and I'm seeing these repeated errors in the pod logs. "7 workers.go:103] error syncing 'model-development/a8spkcsnvc8dt799rkjb Workflow[] failed. ErrorRecordingError: failed to publish event, caused by: authentication error! Original Error: <nil>, Auth Error: failed to issue token. Error: failed to get token: oauth2: "invalid_client" "Client authentication failed (e.g., unknown client, no client authentication included, or unsupported authentication method)." I'm not sure how to resolve this. My helm auth values seem good to me. If I were to have corrected this, would these errors automatically resolve or would they continue to error if initiated with faulty config prior?

Hey Logan, so auth errors should go away if config in both sides (Flyte and the IdP) are right. So this log coming from the worker (handled by flytepropeller) failing to publish an event to flyteadmin due to auth errors. I guess your setup uses the internal authz server right?

I'm not totally sure how the internal server works to be honest. I currently have oidc, and internal configured, but I think I need to add flyteclient, and that's why it isn't working?

So from this section https://docs.flyte.org/en/latest/deployment/configuration/auth_setup.html#apply-oidc-configuration What

internal

means here is propeller to admin

Ok, I think I may have solved that, my hash that I was getting from a different command was giving me a different value than from the command listed on that page

Cool In regards to RUNNING state, take a look the the timeline in the UI. It should give you a hint on where the bottleneck is

Thanks!