I already searched the history and I am pretty sure, that it is not available from what I have read, but maybe you can confirm on this. We would like to restrict users accessing our Flyte installation to only certain projects. So a user should not see all projects, but only those she is assigned to. Basically, we need an authorization concept. Authentication is already built in to Flyte. But I could not find anything related to authorization e.g. permissions for the software. Answers appreciated 🙂

This would be really good feature and that will also help. But for now we would be doing this by creating a separate cluster for each project.

Authorization is not available in Flyte. If you see our reasoning is, there is no standard way. You can use union cloud @Vipul Goswami you as well Union cloud has an opinionated implementation using a 3p provider

@Ketan (kumare3): Do you have any example on how to make authentication work with pyflyte? I just activated authentication for the console, which works fine. I am trying to schedule a workflow execution with pyflyte, but the CLI gets simply stuck for minutes, without continuing or throwing an error. I have stick to the following guide https://docs.flyte.org/en/latest/deployment/configuration/auth_setup.html#authentication-setup

With pyflyte it should just work

That's how it looks. It just remains there, doing nothing - no feedback at all. I have to cancel it with Ctrl+c

Why is it insecure

It is insecuret because we have not yet enabled TLS encryption. Why is it not possible to authenticate over SSH Port forwarding?

@Rob Ulbrich what are you using for auth server? the internal one or an external provider?

We currently use the internal auth server which then sends authentication requests to an external auth provider. I can not tell 100% percent, because to me the documentation, like very often in Flyte, is quite confusing

@Rob Ulbrich what David means is if you refer to the diagram on this page - Authentication providers like Google do not implement complete Oauth2 protocol. For cases like this Flyte provides an authorization server. On the other hand services like Auth0, Okta and many others provide their own Authorization server in which the flyte provided server should be disabled

Is there any full reference how to use only the internal authentication server? What I could find was a page, where some code examples were provided, that can be copied, but does not explain about the meanings of the single settings. Also it is not explained what different options are there

@Rob Ulbrich I'm sorry and agree that several portions of the Flyte docs can have better explanations, this is in part a shared struggle and an opportunity for anyone to contribute. If you want to use the internal auth server only, this is what you should do: 1. As described here, even with the internal server, you still need a provider for OIDC (Identity) 2. Depending on your provider, you'd complete the steps to configure the identity layer and apply the changes With that, you should have both the UI and CLI covered by an auth flow In any case, please let us know if you have questions.

I have followed exactly those steps. The UI authentication is fine, but the CLI authentication is a mystery to me. Actually I would also like to use the OIDC provider for the CLI as well. But it looks like that only the internal auth server is used

Cc @Haytham Abuelfutuh can you chime in here too

I'll have a call with Rob and will report back here any outstanding question

Thank you, @David Espejo (he/him) @Rob Ulbrich, the default CLI behavior requires it to open a browser window for you to authenticate (since it's trying to authenticate the user). Is that something available on the machine you are authenticating on? There are a few different modes to authenticate if that's not available (remote access into a headless server... etc.). Can you rerun the same command with this env var set?

export FLYTE_SDK_LOGGING_LEVEL=10
pyflyte....

Can you try a different auth mode:

admin:
  # For GRPC endpoints you might want to use dns:///flyte.myexample.com
  endpoint: dns:///127.0.0.1:8089/
  authType: DeviceFlow
  insecure: true
logger:
  show-source: true
  level: 0

In the device flow mode, it should output a URL (with code embedded in the URL) for you to copy and run on a machine that has a browser, once authenticated, the CLI session will automatically pick that up and continue... If you are trying to run an automation scenario (CI for example) where there will be no user involved, there is ClientCredentials mode that can be used... please let us know if that's what you are trying to do for further explanation 🙂 Also 100% on the docs lacking, I take responsibility of that. We do need yours and everyone in the community who have setup Auth to help build a better version of this.

Current status: • PKCE flow works • `DeviceFlow`is to-be-tested once the current token expires. Results from here will likely lead to other questions. Outstanding questions: 1. In the configuration for

flyte-core

in this section, what's the difference between

clientId: flytectl

and

selfAuthServer.staticClients.flyte-cli

? (see code) a. David's guess is that the first one, under

thirdPartyConfig

configures the client to interact with the external IdP, and the remaining

staticClients

will interact with the internal auth server to obtain the tokens and complete the flows. b. Shouldn't one of these be called `flyte-ui`to avoid confusions? c. Why those two specific callback addresses? Especially this one that looks a bit arbitrary:

- <http://localhost:12345/callback>

2. Is it possible to reset the auth token manually from Flyte?

Great questions! Can you elaborate on what made PKCE work, @David Espejo (he/him)? 1. I don't think the link for the section is correct, what should it point to? a. thirdPartyConfig is more or less a convenience. Flyte exposes an API that is anonymously accessible that allows CLIs to "discover" the clientId they should use to authenticate (external or internal is orthogonal). The reason is, we wanted to eliminate any config requirements on the client side to correctly authenticate. Once the administrator sets up Auth, clients will automatically discover that and start behaving correctly. b. If using the built-in OAuth Authorization Server (it should really be called Authentication Server),

thirdPartyConfig

should, therefore, refer to a client that exist within the

staticClients

list. c. If using an external OAuth Authorization Server,

staticClients

is naturally completely irrelevant and

thirdPartyConfig

should refer to a client that you have created as Native App OAuth Client in that external OAuth Authorization Server. d. The client used by "the flyte ui" to authenticate is listed under oidc. The reason it's not called flyte UI is that it's only really only used to authenticate to the UI... When you authenticate to the CLI (see this diagram) it also uses the same client... 2. The access token the CLI obtained? it's stored in key-ring (or equivalent in windows/linux)... on OSX, you can open. KeyChain and find

flyte_at

and delete that entry...

Can you elaborate on what made PKCE work, @David Espejo (he/him)?

Besides the good work from Rob, their Pkce flow works on a machine with a GUI and a browser. In a GUI-less machine it doesn't work but no error logs are generated, it just remains waiting to invoke a browser that, well, it's not there. The other difference between the two machines is the Python version (works on 3.10, not on 3.11) but I'm not sure that's connected.

Great, that was my guess too above, @David Espejo (he/him) ☝️ please take a look at the explanation and what to do next...

@David Espejo (he/him) for headless mode - you should use potentially the client flow?

@Haytham Abuelfutuh thank you. It's a great explanation and will look to adapt and add to the docs. @Rob Ulbrich does the above explanations answer your question? Did DeviceFlow work in your headless VM?

@David Espejo (he/him) I can only 50% follow the explanations to be honest