Hey, as an extra layer of security I want to authenticate users in the ALB by adding oidc Ingress annotations. How would I configure `flytectl`to still be able to authenticate users when the OIDC flow happens in the ALB rather than in the Flyte Console? On another note, is it possible to deploy Flyte internally and still be able to authenticate users/deploy workflows in CI/CD? I guess the topics are quite different, but it would be interesting to see what choice you prefer and how that is implemented in practice. Thanks 🙂

i thought flytectl should still work if you are using standard pkce flow? cc @Haytham Abuelfutuh

I am not able to get it to work, using

authType: Pkce

and

insecure: false

I am getting following error when trying to communicate with the backend. The 302 response is the HTTP response from login page to IdP, as the oidc flow is happening in the ALB rather than in the application itself. For the second question I was referring to this security advisory where the suggested workaround is disabling flyte console availability on the internet. Not sure how that would work in practice, i.e., how do you do communicate with the Flyte Admin either locally or from a CI/CD workflow?

~: flytectl get projects
Error: rpc error: code = Unknown desc = Found: HTTP status code 302; transport: received the unexpected content-type "text/html"

@Hampus Rosvall you are no longer vulnerable to that

Yes, that’s awesome! But for security reasons it would be really nice adding an additional security layer in the ALB. Have you done something like that previously?

I think someone else should answer that / my expertise is missing here

Right, no problem! Is there anyone that would know we can loop in?

Ya @Haytham Abuelfutuh or @Prafulla Mahindrakar or @Yee or @Sören Brunk

Hi @Hampus Rosvall why not use the same domian name that you use for flyteconsole in flytectl config.yaml and have the ingress handle the redirection for you . Also the ssl termination happens at the elb which has ingress rules for both flyteconsole which uses REST endpoints and for flytectl which uses GRPC . Flyteadmin doesn’t itself come up with ssl and relies on ssl termination to happen in the upper layers. If you have tried this and it fails please check the ingress is getting applied correctly . There should be two ingress in flyte name space one for grpc and other for non-grpc For your second question , you can look at how we run our functional tests using flytectl https://github.com/flyteorg/flytetools/blob/d4d9c38d99cf8ba7648783bf417448ffe19d683d/functional-tests/config.yaml#L7 Here it relies on the default authType of client_secrets https://github.com/flyteorg/flyteidl/blob/654427ef4027c6b48d997abe8770c2e8b915e84a/clients/go/admin/config.go#L32 (default 0)

Hi Prafulla, I have tried to modify the domain name for the flyteconsole in the

~/.flyte/config.yaml

but I am getting the following error below as the are using the oidc Ingress annotations i.e., the endpoint hits the

~: flytectl get projects
{"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [storage] updated. No update handler registered.","ts":"2022-05-12T10:11:43+02:00"}
{"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [root] updated. No update handler registered.","ts":"2022-05-12T10:11:43+02:00"}
{"json":{"src":"viper.go:400"},"level":"debug","msg":"Config section [admin] updated. Firing updated event.","ts":"2022-05-12T10:11:43+02:00"}
{"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [files] updated. No update handler registered.","ts":"2022-05-12T10:11:43+02:00"}
{"json":{"src":"client.go:183"},"level":"error","msg":"failed to initialize token source provider. Err: failed to fetch auth metadata. Error: rpc error: code = Unknown desc = Found: HTTP status code 302; transport: received the unexpected content-type \"text/html\"","ts":"2022-05-12T10:11:44+02:00"}
{"json":{"src":"client.go:188"},"level":"warning","msg":"Starting an unauthenticated client because: can't create authenticated channel without a TokenSourceProvider","ts":"2022-05-12T10:11:44+02:00"}
{"json":{"src":"client.go:64"},"level":"info","msg":"Initialized Admin client","ts":"2022-05-12T10:11:44+02:00"}
Error: rpc error: code = Unknown desc = Found: HTTP status code 302; transport: received the unexpected content-type "text/html"
{"json":{"src":"main.go:13"},"level":"error","msg":"rpc error: code = Unknown desc = Found: HTTP status code 302; transport: received the unexpected content-type \"text/html\"","ts":"2022-05-12T10:11:44+02:00"}

As we are using oidc ALB annotations i.e.,

<http://alb.ingress.kubernetes.io/auth-type|alb.ingress.kubernetes.io/auth-type>: oidc

and pointing to our IDP.

Flyte does support OIDC based authentication and OAUTH 2 server through flyteadmin directly https://docs.flyte.org/en/latest/deployment/cluster_config/auth_setup.html And any reason you are not using this. Using the ALB sounds interesting for doing OIDC and offloading this from flyteadmin. Given that case are you bringing up flyteadmin with auth i.e is useAuth property set to true in config. Given an offloaded process of doing auth, then admin shouldn’t be enabled for this. We haven’t tried such a setup but would be interesting to see what all it takes to make it work but first would recommend using the flyteadmin auth route to get started. With flyteadmin based auth, you automatically get audit info at flyte entity level, eg : who created the execution, With the ALB based OIDC i am not sure how this user info gets available to admin . Since we store in auth context such info

Hi Prafulla, thanks for quick response! The main reason for doing the OIDC authentication in the ALB is security, we have used FlyteAdmin for the auth flow previously but an extra layer of security/off loading auth to the ALB would be really nice. The useAuth property is thus false currently

Hi @Hampus Rosvall you might want to check if the annotation(

<http://alb.ingress.kubernetes.io/auth-type|alb.ingress.kubernetes.io/auth-type>: oidc)

you added exists on both the ingress objects. for grpc helm installer creates a new ingress object

kubectl get ingress -n flyte

Yes, both

flyte-core

and

flyte-core-grpc

has the same annotations

I think admin is forcing the auth here, We should probably add authType None which is missing in admin auth config for cases like yours

Yes, I am using port-forwarding currently. But was curious to see if we could move away from that and do some special configuration point

flytectl

to our endpoint and still be able to handle authentication. But perhaps there is no way to do so?

None atm. But we should figure this out . Do you mind opening an issue for this .

Totally missed your reply. Yes of course I can do that 🙂