Hi Flyte Fellows, Has anyone used Auth0 or Okta with Flytectl before? I followed the Okta instructions via OIDC and was able to verify Auth works with Auth0 and Okta for Flyte Console. But after following instructions for flytectl I am still getting following error:

flytectl get project 
Error: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: x509: certificate signed by unknown authority"

cc: @Prafulla Mahindrakar

I think you have self signed or cert signed by unknow authority. You can choose to skip the verification of the cert using

insecureSkipVerify

flag https://github.com/flyteorg/flyteidl/blob/7b1fd01ff429d28755765204f00dea57598d8851/clients/go/admin/config.go#L41 But its not recommended for prod usecases and should be used only for testing.

Yes, I was aware of this flag. But since we are evaluating Flyte for prod usecases, I would like to make sure what is not right when it comes to the certificate and sign an appropriate one for flytectl and flyte console both.

How did you provision your certificate and which certifying authority is being used to sign it.

Hi @Prafulla Mahindrakar I think I changed my cert config and now above error is gone. I am now getting a different error:

root@0d4683ee86d5:/# flytectl get project
Error: rpc error: code = Unauthenticated desc = token parse error [JWT_VERIFICATION_FAILED] Could not retrieve id token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken

Hi @Shahwar Saleem by default flytectl uses pkce which doesn’t require to store shared client secrets and it requires browser access to complete the auth flow. It will connect you to configured OIDC provider and complete the flow. It seems you might be on a machine which doesn’t have browser access, If thats the case then you can change the authType from pkce to client secret eg : config.yaml or you can pass these flags from cli too

admin:
  # For GRPC endpoints you might want to use dns:///flyte.myexample.com
  endpoint: dns:///flyte.org
  # Change insecure flag to ensure that you use the right setting for your environment
  insecure: false
  clientId: *********
  authType: ClientSecret
  clientSecretLocation: /home/runner/secret_location
logger:
  # Logger settings to control logger output. Useful to debug logger:
  show-source: true
  level: 1

Auth Update on this thread. @Prafulla Mahindrakar Did implement Device Authentication Flow for flytectl under the same PR. Here are some details after that: • I was able to authenticate using Device Flow on the browser by making following changes: ◦ Configured Flyte Admin Config Map according to this ◦ Needed to add

metadataUrl

to be

.well-known/openid-configuration

. This is a suffix to baseURL internally. ◦ Needed to add new clientId which was provided by Auth Server App. ◦ Needed to change the flyteadmin image to a new one corresponding to DeviceFlow changes:

<http://ghcr.io/flyteorg/flyteadmin:v1.1.37-deviceauth|ghcr.io/flyteorg/flyteadmin:v1.1.37-deviceauth>

. This change needs to happen in flyteadmin deployment accross all occurrences of the flyteadmin image. ◦ Needed to change FlyteCTL config to something like this:

admin:
  endpoint: <URL>
  authType: DeviceFlow
  insecure: false
  caCertFilePath: <certfilepath>
  deviceFlowConfig:
    deviceFlowTimeout: 200s
logger:
  show-source: true
  level: 20

After authenticating, I am getting an error which does signify there is an issue with JWT response. Apparently, for our usecase an

audience

needs to be specified while requesting auth from Auth0 server. Currently,

flyteadmin

is not using audience field and due to missing Audience, Auth Server is not able to complete the Device Authentication flow, because the token after authorization is a default one, which Auth server is not accepting. A few things: • We would need full Auth support including Audience field to fully utilize Authentication with Flytectl. • The team is now outside POC and implementing a full Flyte Deployment which will need full Authentication capability for Flyteconsole as well as FlyteCTL. CC: @Andrew Achkar @Geoff Salmon @Haytham Abuelfutuh

This is great to hear @Shahwar Saleem thank you for the update! The goal is to make this very intuitive to use/configure. I would really appreciate it if you can help us with the Auth0 config part (to add another section under the auth doc for Auth0)… I would like to understand a big deeper the missing audience problem. Is this something that can be addressed in the request for a token either on the flytectl side or admin? is it a configuration issue/change in Auth0? Also why is there a need for setting caCertFilePath? is it because your host doesn’t have a publicly accessible/verifiable SSL Cert?

So it seems auth0 is always sending opaque token on device flow path though admin flow require a JWT and parser fails here in admin https://github.com/flyteorg/flyteadmin/blob/master/auth/authzserver/resource_server.go#L33 From Auth0 community pages it seems its recommended to pass in the audience param when requesting an access token inorder to receive a JWT https://community.auth0.com/t/why-is-it-necessary-to-pass-the-audience-parameter-to-receive-a-jwt/11412/6 But the doc also do mention configuring a default audience parameter on the auth0 auth server. This is how i tested my changes using okta which worked. Wanted to check why can’t auth team make this change . If we set it at the auth server side , the token issued will be verified by admin using the allowedAudience field and will accept or reject the token based on if the token is intended for it or not. Now coming to another point that clientCredentials flow works with same auth server and flyteconsole is able to pull resources from flyteadmin resource server. And browsing through the golang aouth library and tracing our code path, I couldn’t find that we are passing the aud info in that flow. So wondering how come clientCreds work then. The config used for clientCreds use this and this doesn’t pass audience field https://github.com/flyteorg/flyteidl/blob/master/clients/go/admin/token_source_provider.go#L156-L161 Oauth library expects any additional params eg audience to be passed in

// EndpointParams specifies additional parameters for requests to the token endpoint.
	EndpointParams url.Values

https://github.com/golang/oauth2/blob/master/clientcredentials/clientcredentials.go#L44 which gets used here https://github.com/golang/oauth2/blob/master/clientcredentials/clientcredentials.go#L97 Sample test with aud https://github.com/golang/oauth2/blob/master/clientcredentials/clientcredentials_test.go#L25 But none of this config is passed from flyteadmin which is why i am uncertain that adding the aud field is going to solve the device flow issue. Wanted to test this with a custom build and verify before making a configurable change. Let me know what you all think .

@Prafulla Mahindrakar Please let me know if there is any update on this or how are the timelines looking for Audience.

@Shahwar Saleem did we get a chance to check if adding the aud on auth server works. You can use the following draft pr in flyteidl for passing the audience from cli https://github.com/flyteorg/flyteidl/pull/314 . • Create new flytectl binary using it

go mod edit -replace <http://github.com/flyteorg/flyteidl=github.com/flyteorg/flyteidl@4b475700a8df0e7736fed995ee84a4cc22e453fd|github.com/flyteorg/flyteidl=github.com/flyteorg/flyteidl@4b475700a8df0e7736fed995ee84a4cc22e453fd>
 go mod tidy

And then run using the audience by passing the right aud field for your app

flytectl get projects --logger.level=6  --admin.authType DeviceFlow  --admin.deviceFlowConfig.audience <aud>

Ref used on how to pass aud https://auth0.com/docs/secure/tokens/access-tokens/get-access-tokens