Hello. Thanks to the Flyte team for their work on ...
# ask-the-community
t
Hello. Thanks to the Flyte team for their work on this framework. Real quick, using Flyte-Binary as a helm chart onto AWS EKS. Trying to set up OIDC and/or OAuth2 via Auth0 and have been messing around with some configs but have been unsuccessful getting the
/console
endpoint to query the browser for login with variations off of the following values.yaml config:
Copy code
auth: 
      enabled: true
      enableAuthServer: false
      oidc:
        baseUrl: https://{DOMAIN}.<http://auth0.com/.well-known/openid-configuration|auth0.com/.well-known/openid-configuration>
        clientId: {CLIENT ID}
        clientSecret: {CLIENT SECRET}
      authorizedUris:
        - https://{LOAD BALANCED ADDRESS}.com
        - https://{LOAD BALANCED ADDRESS}.com
Bit unsure how to transpose the documentation from the official site into
flyte-binary
as the documentation seems to only go over
flyte-core
or
flyte
. The documentation doesn't seem to really exist for flyte-binary and there's scant discussion surrounding the topic unfortunately. I understand on some level that there are three components to the Flyte deployment but how does one access those configurations through the Helm chart within the
flyte-binary
deployment? Specifically, how are you supposed to affect the various OIDC/auth config values when the templates for binary don't seem to match the documentation? Also, we wish to use Auth0 as our external authorization server and don't need the internal server, but there have been conflicting resources suggesting that internal still needs to be defined. Any help in the matter would be most appreciated.
y
internal is not needed (and should not be specified) if using external
this is basically what we use internally for binary - but we use the internal auth server, so you’ll need to track down through the helm chart what values to set.
@Tommy Nam - i copied that snippet to a github discussion just in case slack removes history.
if you have any more hints or comments about auth0 that you realized, can you add them there please?
t
@Yee (EDIT: If the following post is asking for too much assistance, then that is alright - we have started to consider using the internal auth that comes with the default Helm chart for now and closing off access through subnets and security groups instead. Though at some point, we would like to productionize Helm with Auth0 but that can be addressed at a later time.) Yes, if possible, and if you or the team has the time, we would like some more clarification around passing in the
audience
parameter during a
pyflyte
command (Basically, the auth). I've looked at previous discussions on this slack channel and google, and it appears that this was an issue with auth0 that has appeared in the past and I can see some changes were made to flyteidl. Even after specifying the audience explicitly as well as trying to retrieve it from FlyteAdmin, it doesn't seem to work.
.flyte/config.yaml
Copy code
admin:
  # For GRPC Endpoints you might want to use dns:///flyte.example.com
  #endpoint: dns:///localhost:8080
  endpoint: dns:///<endpoint here>
  authType: ClientSecret
  clienttId: <Client ID>
  #clientSecretLocation: location
  clientSecretEnvVar: FLYTE_CREDENTIALS_CLIENT_SECRET
  useAudienceFromAdmin: true
  audience: https://<audience>.<http://jp.auth0.com/api/v2/|jp.auth0.com/api/v2/>
Here is our config.yaml at the moment, but we run into issues where
pyflyte run
will say this:
flytekit.clients.auth.exceptions.AuthenticationError: Status Code (403) received from IDP: {"error":"access_denied","error_description":"No audience parameter was provided, and no default audience has been configured"}
So this is obviously Auth0 requesting an audience parameter in the request and it seems like maybe
pyflyte
and/or
flytekit
doesn't send audience by default. Essentially we're wondering how to rectify this issue within our infra. One possible contributing factor to this may be the pyflyte/flytekit env vars, but we are a little unclear on how to configure the client credentials flow as listed here using auth0: https://docs.flyte.org/en/latest/deployment/configuration/auth_setup.html#continuous-integration-ci
env vars we export into shell in dev env
Copy code
# When using OAuth2 service auth, this is the username and password.
export FLYTE_CREDENTIALS_CLIENT_ID=client id
export FLYTE_CREDENTIALS_CLIENT_SECRET=client secret

#export FLYTE_CREDENTIALS_CLIENT_SECRET_FROM_FILE="~/.flyte/something here"

# This tells the SDK to use basic authentication. If not set, Flytekit will assume you want to use the
# standard OAuth based three-legged flow.
export FLYTE_CREDENTIALS_AUTH_MODE=basic                        <---------------------------------------should this be unset for auth0?

# This value should be set to conform to this
# `header config <<https://github.com/flyteorg/flyteadmin/blob/12d6aa0a419ccec81b4c8289fd172e70a2ded525/auth/config/config.go#L124-L128>>`_
# on the Admin side.
export FLYTE_CREDENTIALS_AUTHORIZATION_METADATA_KEY=auth_metadata_key             <-------------------------------------------------------------- not sure how this corresponds to auth0 at the moment

# When using basic authentication, you'll need to specify a scope to the IDP (instead of ``openid``, which is
# only for OAuth). Set that here.
export FLYTE_CREDENTIALS_OAUTH_SCOPES=

# Set this to force Flytekit to use authentication, even if not required by Admin. This is useful as you're
# rolling out the requirement.
export FLYTE_PLATFORM_AUTH=True
In particular, the
FLYTE_CREDENTIALS_AUTHORIZATION_METADATA_KEY
. We're thinking this is potentially where to pass the audience in? But we are not sure. Any and all help would be greatly appreciated in this matter and sorry if these questions seem rather rudimentary. Our team is rather new to Flyte and we are trying to grokk it as best we can. Addendum: here is our configmap template for auth in case it helps:
Copy code
data: |
  auth:
    userAuth:
      openId:
        clientId: AUTH0_CLIENT_ID
        baseUrl: AUTH0_BASE_URL
        scopes:
          - profile
          - openid
          - offline_access
          - client_credentials
    appAuth:  
      authServerType: External
      externalAuthServer:
        baseUrl: AUTH0_BASE_URL
        metadataUrl: .well-known/openid-configuration
        allowedAudience: AUTH0_AUDIENCE
      thirdPartyConfig:
        flyteClient:
          clientId: AUTH0_CLIENT_ID
          redirectUri: <http://localhost:53593/callback>
          audience: AUTH0_AUDIENCE
          scopes:
            - offline
            - all
    authorizedUris:
      - AUTH0_AUDIENCE
      - LOADBALANCER_URL
      - AUTH0_BASE_URL
      - <http://LOADBALANCER_GRPC_URL>
      - <https://LOADBALANCER_GRPC_URL>
      - <http://flyteadmin:80>
      - <http://flyteadmin.mlops-services.svc.cluster.local:80>
      - <http://HELM_NAME-flyte-binary.flyte.svc.cluster.local:80>
      - <http://HELM_NAME-flyte-binary.flyte.svc:80>
      - <http://HELM_NAME-flyte-binary.flyte:80>
      - <http://HELM_NAME-flyte-binary:80>
  server: 
    security:
      secure: false
      useAuth: true
y
let’s talk on monday… i don’t think you should give up
FLYTE_CREDENTIALS_AUTHORIZATION_METADATA_KEY
is something else. i think you can ignore that for now. i forget what the default is, you need to just set it to
authorization
but not looking at code right now, might be wrong, but i’m pretty sure it’s not the audience
what flow are you trying to get working? is this a user doing it? or is this a client id/client secret flow?
t
This is a user using pyflyte. When we try to use the
Pkce
workflow, we authenticate successfully with Auth0 but get met with an
Errno98 address already in use
. For whatever reason, there are two oAuth requests being sent to localhost:53593, and the browser opens up two tabs both pointing at localhost:53593 but with two different callback codes in the URLs. And when we use the
ClientSecret
flow, we get met with an
audience not specified
error. Which is coming from Auth0 not receiving an "audience" parameter when the Flyte service requests a token
One alternative we have considered is using Okta instead of Auth0, so that could be one way to get around this issue
y
and when testing the second case, you configured admin to return the audience?
l
not sure if its the cause, but you have a typo in
.flyte/config.yaml
,
clienttId
instead of
clientId
t
@Laura Lin Thanks for pointing that out! It turned out to be something else entirely, and we ended up making a fork for internal use
d
Hi Tommy, we are also experiencing this issue with
address already in use
when using the
Pkce
flow, could you please share how you got passed it?
127 Views