fancy-plumber-70674
05/10/2023, 11:10 AMpyflyte run
command with an external auth server? I can see that it is part of the flyteIdl repository but have not worked with Go or gRPC in the past - though I am open to trying nonetheless.
We are receiving 403 forbidden errors due to the flyte-binary
pod/deployment being unable to send the audience
parameter. I am assuming that this is between FlyteAdmin and FlytePropeller, though I could be wrong.
So essentially, the flow is like this:
• Localhost/client/machine sends pyflyte run --remote
command to gRPC backend - AWS ALB w/ SSL/TLS
• Auth request is successful for pyflyte/flytekit - using Auth0 as external auth server
• Web console registers and then displays workflow with UNKNOWN status
• No Pods that were requested in the pyflyte command are scheduled
• Inspection of flyte-binary deployment w/ kubectl shows that a missing audience
parameter is needed
• Tons of requests with 403 - seems retry logic never stops - Necessitates killing deployment
• Auth0 Logs show that all the requests fail due to audience
Relevent Github issue is here with further logs/details: https://github.com/flyteorg/flyte/issues/3662
Any assistance in the matter would be greatly appreciated. We believe that this is the final step in getting Auth0 working with a flyte-binary deployment and would be more than glad to provide supporting documentation/code from forks if need be.freezing-airport-6809
fancy-plumber-70674
05/11/2023, 2:47 AMflyte-binary
) manually through kubectl apply
or kubectl patches
for the various resources? For example, we can override the configmap for auth for flyteadmin by pulling the manifest from EKS, parsing and patching certain blocks/values, then reapplying it, etcfancy-plumber-70674
05/11/2023, 7:12 AMfreezing-airport-6809