https://flyte.org logo
#ask-the-community
Title
# ask-the-community
t

Tommy Nam

05/10/2023, 11:10 AM
Hello community, does anybody have any insight into modifying the gRPC requests that go on in the backend after a successfully authenticated
pyflyte run
command with an external auth server? I can see that it is part of the flyteIdl repository but have not worked with Go or gRPC in the past - though I am open to trying nonetheless. We are receiving 403 forbidden errors due to the
flyte-binary
pod/deployment being unable to send the
audience
parameter. I am assuming that this is between FlyteAdmin and FlytePropeller, though I could be wrong. So essentially, the flow is like this: • Localhost/client/machine sends
pyflyte run --remote
command to gRPC backend - AWS ALB w/ SSL/TLS • Auth request is successful for pyflyte/flytekit - using Auth0 as external auth server • Web console registers and then displays workflow with UNKNOWN status • No Pods that were requested in the pyflyte command are scheduled • Inspection of flyte-binary deployment w/ kubectl shows that a missing
audience
parameter is needed • Tons of requests with 403 - seems retry logic never stops - Necessitates killing deployment • Auth0 Logs show that all the requests fail due to audience Relevent Github issue is here with further logs/details: https://github.com/flyteorg/flyte/issues/3662 Any assistance in the matter would be greatly appreciated. We believe that this is the final step in getting Auth0 working with a flyte-binary deployment and would be more than glad to provide supporting documentation/code from forks if need be.
k

Ketan (kumare3)

05/10/2023, 1:34 PM
Yes you will have to configure flytepropeller client to have the same audience parameters, in tge helm chart
t

Tommy Nam

05/11/2023, 2:47 AM
Thank you for the reply. Do you happen to have links to the relevant resources that highlight how to assign the audience parameter to flytepropeller in flyte-binary's Helm chart? I've been looking through flyte-core (not sure if it's a dependency of the flyte-binary chart? so not sure if we can do a sub-chart value override) and the flyte-binary Helm charts and can't seem to find a setting that reflects this for propeller - either in the templates or values Perhaps we are looking in the wrong places, but alternatively, if you happen to know where this setting might be set/affected (in
flyte-binary
) manually through
kubectl apply
or
kubectl patches
for the various resources? For example, we can override the configmap for auth for flyteadmin by pulling the manifest from EKS, parsing and patching certain blocks/values, then reapplying it, etc
May have found solution, will close issue when finalized
Solved, PR to add audience in the internal field to 000-core.yaml template might help future users
k

Ketan (kumare3)

05/11/2023, 4:41 PM
cc @jeev / @Yee
74 Views