Hi Team, We have our flyte cluster url enabled with IAP (Identity-Aware Proxy). While the UI access to flyte cluster works fine using the google account login but accessing flyte cluster using flytectl fails at authentication and produce below error

Error: authentication error! Original Error: rpc error: code = Unauthenticated desc = Invalid IAP credentials: empty token, Auth Error: failed to initialized token source provider. Err: failed to fetch auth metadata. Error: rpc error: code = Unauthenticated desc = Invalid IAP credentials: empty token

Anyone knows how to deal with this? Is there a way to provide the identity token which can be used by flytectl, pyflyte, and flytekit_sdk for authentication

Did you set up a config.yaml in a user-created

/.flyte

directory?

yes

I don't have experience with IAP specifically as we use auth0, but here's an example of what we use for some of our dev users

admin:
  endpoint: 
  authType: ClientSecret
  clientId: ....
  clientSecretLocation: /path/to/secret
  useAudienceFromAdmin: true
logger:
  show-source: false
  level: 5

you might need to specify authType and some sort of clientsecret EnvVar or Location explicitly

Something like this might help. If there is a way where combination of a authType and corresponding token can be provided in configurations

i got the similar issue, deployed flyte on my local cluster, i can visit the web console ,but can not use the flytctl command line , seems like the rpc server is not working

[root@gpu2 .flyte]# flytectl get projects
{"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [storage] updated. No update handler registered.","ts":"2023-07-18T02:43:07-04:00"}
{"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [root] updated. No update handler registered.","ts":"2023-07-18T02:43:07-04:00"}
{"json":{"src":"viper.go:400"},"level":"debug","msg":"Config section [admin] updated. Firing updated event.","ts":"2023-07-18T02:43:07-04:00"}
{"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [files] updated. No update handler registered.","ts":"2023-07-18T02:43:07-04:00"}
{"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [console] updated. No update handler registered.","ts":"2023-07-18T02:43:07-04:00"}
{"json":{"src":"client.go:63"},"level":"info","msg":"Initialized Admin client","ts":"2023-07-18T02:43:07-04:00"}
{"json":{"src":"auth_interceptor.go:86"},"level":"debug","msg":"Request failed due to [rpc error: code = Unavailable desc = connection closed before server preface received]. If it's an unauthenticated error, we will attempt to establish an authenticated context.","ts":"2023-07-18T02:43:07-04:00"}
Error: Connection Info: [Endpoint: dns:///flyte.nginx.k8s:31120, InsecureConnection?: true, AuthMode: Pkce]: rpc error: code = Unavailable desc = connection closed before server preface received
{"json":{"src":"main.go:13"},"level":"error","msg":"Connection Info: [Endpoint: dns:///flyte.nginx.k8s:31120, InsecureConnection?: true, AuthMode: Pkce]: rpc error: code = Unavailable desc = connection closed before server preface received","ts":"2023-07-18T02:43:07-04:00"}

Hi @gifted-train-81198 Could you share: 1. How did you install Flyte? *eg (

flyte-binary

or

flyte-core

) 2. Is the plan to use Flyte's internal auth server for clients (flytectl, etc) and IAP only for OIDC (console)? Or IAP for everything 3. I guess your config.yaml file is missing the auth type, in this case:

authType: Pkce

@average-finland-92144 1. Flyte is installed via helm charts on a k8s cluster behind a vpn 2. I believe we have to use IAP for everything as required by our infosec team

Thank you @adorable-australia-90343 Right now, there's this planned update to the auth docs that could give you some hints. It doesn't include instructions on how to use IAP as authorization server though. Also there's a previous thread where some aspects specific to IAP were discussed: https://flyte-org.slack.com/archives/CP2HDHKE1/p1688147436819889 Please let us know if this is somehow useful for you