Hello. Thanks to the Flyte team for their work on this framework. Real quick, using Flyte-Binary as a helm chart onto AWS EKS. Trying to set up OIDC and/or OAuth2 via Auth0 and have been messing around with some configs but have been unsuccessful getting the

/console

endpoint to query the browser for login with variations off of the following values.yaml config:

auth: 
      enabled: true
      enableAuthServer: false
      oidc:
        baseUrl: https://{DOMAIN}.<http://auth0.com/.well-known/openid-configuration|auth0.com/.well-known/openid-configuration>
        clientId: {CLIENT ID}
        clientSecret: {CLIENT SECRET}
      authorizedUris:
        - https://{LOAD BALANCED ADDRESS}.com
        - https://{LOAD BALANCED ADDRESS}.com

Bit unsure how to transpose the documentation from the official site into

flyte-binary

as the documentation seems to only go over

flyte-core

or

flyte

. The documentation doesn't seem to really exist for flyte-binary and there's scant discussion surrounding the topic unfortunately. I understand on some level that there are three components to the Flyte deployment but how does one access those configurations through the Helm chart within the

flyte-binary

deployment? Specifically, how are you supposed to affect the various OIDC/auth config values when the templates for binary don't seem to match the documentation? Also, we wish to use Auth0 as our external authorization server and don't need the internal server, but there have been conflicting resources suggesting that internal still needs to be defined. Any help in the matter would be most appreciated.

internal is not needed (and should not be specified) if using external

@thankful-minister-83577 (EDIT: If the following post is asking for too much assistance, then that is alright - we have started to consider using the internal auth that comes with the default Helm chart for now and closing off access through subnets and security groups instead. Though at some point, we would like to productionize Helm with Auth0 but that can be addressed at a later time.) Yes, if possible, and if you or the team has the time, we would like some more clarification around passing in the

audience

parameter during a

pyflyte

command (Basically, the auth). I've looked at previous discussions on this slack channel and google, and it appears that this was an issue with auth0 that has appeared in the past and I can see some changes were made to flyteidl. Even after specifying the audience explicitly as well as trying to retrieve it from FlyteAdmin, it doesn't seem to work.

.flyte/config.yaml

admin:
  # For GRPC Endpoints you might want to use dns:///flyte.example.com
  #endpoint: dns:///localhost:8080
  endpoint: dns:///<endpoint here>
  authType: ClientSecret
  clienttId: <Client ID>
  #clientSecretLocation: location
  clientSecretEnvVar: FLYTE_CREDENTIALS_CLIENT_SECRET
  useAudienceFromAdmin: true
  audience: https://<audience>.<http://jp.auth0.com/api/v2/|jp.auth0.com/api/v2/>

Here is our config.yaml at the moment, but we run into issues where

pyflyte run

will say this:

flytekit.clients.auth.exceptions.AuthenticationError: Status Code (403) received from IDP: {"error":"access_denied","error_description":"No audience parameter was provided, and no default audience has been configured"}

So this is obviously Auth0 requesting an audience parameter in the request and it seems like maybe

pyflyte

and/or

flytekit

doesn't send audience by default. Essentially we're wondering how to rectify this issue within our infra. One possible contributing factor to this may be the pyflyte/flytekit env vars, but we are a little unclear on how to configure the client credentials flow as listed here using auth0: https://docs.flyte.org/en/latest/deployment/configuration/auth_setup.html#continuous-integration-ci

env vars we export into shell in dev env

# When using OAuth2 service auth, this is the username and password.
export FLYTE_CREDENTIALS_CLIENT_ID=client id
export FLYTE_CREDENTIALS_CLIENT_SECRET=client secret

#export FLYTE_CREDENTIALS_CLIENT_SECRET_FROM_FILE="~/.flyte/something here"

# This tells the SDK to use basic authentication. If not set, Flytekit will assume you want to use the
# standard OAuth based three-legged flow.
export FLYTE_CREDENTIALS_AUTH_MODE=basic                        <---------------------------------------should this be unset for auth0?

# This value should be set to conform to this
# `header config <<https://github.com/flyteorg/flyteadmin/blob/12d6aa0a419ccec81b4c8289fd172e70a2ded525/auth/config/config.go#L124-L128>>`_
# on the Admin side.
export FLYTE_CREDENTIALS_AUTHORIZATION_METADATA_KEY=auth_metadata_key             <-------------------------------------------------------------- not sure how this corresponds to auth0 at the moment

# When using basic authentication, you'll need to specify a scope to the IDP (instead of ``openid``, which is
# only for OAuth). Set that here.
export FLYTE_CREDENTIALS_OAUTH_SCOPES=

# Set this to force Flytekit to use authentication, even if not required by Admin. This is useful as you're
# rolling out the requirement.
export FLYTE_PLATFORM_AUTH=True

In particular, the

FLYTE_CREDENTIALS_AUTHORIZATION_METADATA_KEY

. We're thinking this is potentially where to pass the audience in? But we are not sure. Any and all help would be greatly appreciated in this matter and sorry if these questions seem rather rudimentary. Our team is rather new to Flyte and we are trying to grokk it as best we can. Addendum: here is our configmap template for auth in case it helps:

data: |
  auth:
    userAuth:
      openId:
        clientId: AUTH0_CLIENT_ID
        baseUrl: AUTH0_BASE_URL
        scopes:
          - profile
          - openid
          - offline_access
          - client_credentials
    appAuth:  
      authServerType: External
      externalAuthServer:
        baseUrl: AUTH0_BASE_URL
        metadataUrl: .well-known/openid-configuration
        allowedAudience: AUTH0_AUDIENCE
      thirdPartyConfig:
        flyteClient:
          clientId: AUTH0_CLIENT_ID
          redirectUri: <http://localhost:53593/callback>
          audience: AUTH0_AUDIENCE
          scopes:
            - offline
            - all
    authorizedUris:
      - AUTH0_AUDIENCE
      - LOADBALANCER_URL
      - AUTH0_BASE_URL
      - <http://LOADBALANCER_GRPC_URL>
      - <https://LOADBALANCER_GRPC_URL>
      - <http://flyteadmin:80>
      - <http://flyteadmin.mlops-services.svc.cluster.local:80>
      - <http://HELM_NAME-flyte-binary.flyte.svc.cluster.local:80>
      - <http://HELM_NAME-flyte-binary.flyte.svc:80>
      - <http://HELM_NAME-flyte-binary.flyte:80>
      - <http://HELM_NAME-flyte-binary:80>
  server: 
    security:
      secure: false
      useAuth: true

let’s talk on monday… i don’t think you should give up

This is a user using pyflyte. When we try to use the

Pkce

workflow, we authenticate successfully with Auth0 but get met with an

Errno98 address already in use

. For whatever reason, there are two oAuth requests being sent to localhost:53593, and the browser opens up two tabs both pointing at localhost:53593 but with two different callback codes in the URLs. And when we use the

ClientSecret

flow, we get met with an

audience not specified

error. Which is coming from Auth0 not receiving an "audience" parameter when the Flyte service requests a token

and when testing the second case, you configured admin to return the audience?

not sure if its the cause, but you have a typo in

.flyte/config.yaml

,

clienttId

instead of

clientId

@numerous-hamburger-7178 Thanks for pointing that out! It turned out to be something else entirely, and we ended up making a fork for internal use

Hi Tommy, we are also experiencing this issue with

address already in use

when using the

Pkce

flow, could you please share how you got passed it?