https://flyte.org logo
Join Slack
Powered by
<@U038FT3LCRE> <@UPTRGR537> Let’s discuss auth her...
# flyte-deployment
h
@Hampus Rosvall @Prafulla Mahindrakar Let’s discuss auth here
h
I just checked the configmaps for propeller and scheduler, there is a portion dedicated to client credentials auth flow, but I can’t seem to find an option to override the scopes here
Copy code
~ k get configmap flyte-{propeller, scheduler}-config  
  admin.yaml: |
    admin:
      clientId: clientId
      clientSecretLocation: /etc/secrets/client_secret
      endpoint: flyteadmin:81
      insecure: true
p
You can add it to the same section with key scopes which takes a list in yaml
h
Right, so something like?
Copy code
admin:
      endpoint: flyteadmin:81
      insecure: true
      clientId: "{{ .Values.secrets.adminOauthClientCredentials.clientId }}"
      clientSecretLocation: /etc/secrets/client_secret
      scopes: ["myScopes"]
p
yes. Or i guess yaml also uses this
Copy code
admin:
      endpoint: flyteadmin:81
      insecure: true
      clientId: "{{ .Values.secrets.adminOauthClientCredentials.clientId }}"
      clientSecretLocation: /etc/secrets/client_secret
      scopes: 
         - myscopes
h
Cool, let’s try that
p
h
Okay some progress, I get sent to Azure IdP and get redirected with
Copy code
Cool! Your authentication was successful and you can close the window.
But I am getting this response now from the admin
Copy code
Error: rpc error: code = Unauthenticated desc = token parse error [JWT_VERIFICATION_FAILED] Could not retrieve id token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken
And in the logs from the admin it finds an existing metadata Bearer, but when I validate the token on jwt.io it says that the signature is invalid. Maybe I have not configured the PKCE app successfully?
h
Hey @Hampus Rosvall do you have time to debug this live now?
h
Ah, not by my computer atm.
But I can try to dig into this more during the day tomorrow and we can have a call tomorrow if it’s not resolved?
h
👍
p
Hi @Hampus Rosvall, can you paste the logs from admin . And also what are the config.yaml for flytectl and your admin auth config. If it easier we can get on a call to debug this
h
Hey, sure. By the way, is there a way to override the allowed audience for the PKCE auth flow?
Copy code
appAuth:
        authServerType: External
        externalAuthServer:
          baseUrl: <https://login.microsoftonline.com/tenant/v2.0>
          metadataUrl: .well-known/openid-configuration
          allowedAudience:
            - clientid
This is what I am doing for the client credentials flow, and I am getting invalid audience error for the PKCE flow now
I think the error is within my app for PKCE flow, and the scopes I am sending - so I am just trying to figure out how it should be configured.
p
In your Azure authorization server can you check what is the audience configuration used. The allowedAudience field needs to match one set in the authorization server
h
Yes, when I validate the token, the audience should equal the clientId.
So I need to find a way to provide that during token validation, as the allowed audience is different for client credentials flow and PKCE due to two different apps
https://github.com/flyteorg/flyteadmin/blob/f72aa348e4a548d38f5afc5dcbc49f56fe67a75f/auth/authzserver/provider.go#L138-L143 seems like the code checks if the audience from the claim is in the list of allowed audiences, so I might be able to add it
p
The allowed audience can be a list and its usually the endpoint which flyte is hosted on And this what should be configured in the allowed audience in admin so that token sent by auth server has the same audience configured in admin
yes exactly .
For eg in okta this config is available directly on the auth server config page . This not specific to an app but the capability of auth server when it rolls out token and who its intended audience is .
h
Copy code
flyteadmin-76f6476884-9vnxq flyteadmin panic: interface conversion: interface {} is string, not []interface {}
flyteadmin-76f6476884-9vnxq flyteadmin
Copy code
allowedAudience:
            - propellerClientId
            - flytectlClientId
p
Can you double quote those values . or use this representation
Copy code
key1: [value1,value2,value3,value4,value5]
h
I can’t right now, have a meeting in 15 min
p
Ok sure. Let me know your availability .
h
I have set up an app according to these docs now. I am getting the following error
Copy code
Tokens issued for the 'Single-Page Application' client-type may only be redeemed via cross-origin requests
which seems to be discussed in this thread e.g., this solution https://stackoverflow.com/a/67280275. Is this something I can add to the request?
p
Can you check if the failure comes after this https://github.com/flyteorg/flyteidl/blob/master/clients/go/admin/pkce/handle_app_call_back.go#L44 And may be you would need to add the origin url when fetching the auth token at this place.
h
Hm, how would I check that? The full error is from here
Copy code
{"json":{"src":"token_source_provider.go:121"},"level":"error","msg":"Error fetching token using auth flow due to error while exchanging auth code due to oauth2: cannot fetch token: 400 Bad Request\nResponse: {\"error\":\"invalid_request\",\"error_description\":\": Tokens issued for the 'Single-Page Application' client-type may only be redeemed via cross-origin requests
p
So after opening the url it redirects to auth server to login and then on the callback hits the app call back hander with authCode to be exchanged for the token The app call back handler exchanges that here https://github.com/flyteorg/flyteidl/blob/master/clients/go/admin/pkce/handle_app_call_back.go#L44 So it seems in your case some policy blocks this exchange . Also looking at this i guess its forbidden to set this Origin header https://stackoverflow.com/questions/46808011/how-to-set-the-origin-request-header And recommended way is to allowCors. I remember we spoke about disabling this to test it out. Is it still disabled in your admin config. (allowCors) . With allowCors do you get this far in the flow or does it fail earlier .
h
This is the server portion of the 
server.yaml
configMap:
Copy code
server:
      grpcPort: 8089
      httpPort: 8088
      security:
        allowCors: true
        allowedHeaders:
        - Content-Type
        allowedOrigins:
        - '*'
        secure: false
        useAuth: true
So the error is still hit here:
Copy code
{"json":{"src":"token_source_provider.go:121"},"level":"error","msg":"Error fetching token using auth flow due to error while exchanging auth code due to oauth2: cannot fetch token:
p
And what about the admin logs , can we check those too for the error.
h
Copy code
flyteadmin-858647d86d-8f4h8 flyteadmin {"json":{"src":"handlers.go:237"},"level":"debug","msg":"Running authentication gRPC interceptor","ts":"2022-09-01T14:23:10Z"}
flyteadmin-858647d86d-8f4h8 flyteadmin {"json":{"src":"token.go:83"},"level":"debug","msg":"Could not retrieve bearer token from metadata rpc error: code = Unauthenticated desc = Request unauthenticated with Bearer","ts":"2022-09-01T14:23:10Z"}
flyteadmin-858647d86d-8f4h8 flyteadmin {"json":{"src":"handlers.go:247"},"level":"info","msg":"Failed to parse Access Token from context. Will attempt to find IDToken. Error: [JWT_VERIFICATION_FAILED] Could not retrieve bearer token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with Bearer","ts":"2022-09-01T14:23:10Z"}
flyteadmin-858647d86d-8f4h8 flyteadmin {"json":{"src":"token.go:103"},"level":"debug","msg":"Could not retrieve id token from metadata rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken","ts":"2022-09-01T14:23:10Z"}
p
The message is clipped can you paste the entire log . Is it the same issue
Copy code
Tokens issued for the 'Single-Page Application' client-type may only be redeemed via cross-origin requests
h
Copy code
{"json":{"src":"token_source_provider.go:121"},"level":"error","msg":"Error fetching token using auth flow due to error while exchanging auth code due to oauth2: cannot fetch token: 400 Bad Request\nResponse: {\"error\":\"invalid_request\",\"error_description\":\"AADSTS9002327: Tokens issued for the 'Single-Page Application' client-type may only be redeemed via cross-origin requests'..}
{"json":{"src":"client.go:188"},"level":"warning","msg":"Starting an unauthenticated client because: error while exchanging auth code due to oauth2: cannot fetch token: 400 Bad Request\nResponse: {\"error\":\"invalid_request\",\"error_description\":\"AADSTS9002327: Tokens issued for the 'Single-Page Application' client-type may only be redeemed via cross-origin requests"...}
{"json":{"src":"client.go:64"},"level":"info","msg":"Initialized Admin client","ts":"2022-09-01T16:23:09+02:00"}
Error: rpc error: code = Unauthenticated desc = token parse error [JWT_VERIFICATION_FAILED] Could not retrieve id token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken
{"json":{"src":"main.go:13"},"level":"error","msg":"rpc error: code = Unauthenticated desc = token parse error [JWT_VERIFICATION_FAILED] Could not retrieve id token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken","ts":"2022-09-01T16:23:10+02:00"}
The dots are just error codes any timestamps from Azure
I am requesting the openid scope btw
p
@Hampus Rosvall I think the app shouldn’t be an SPA but a native app . This is how we use it for Okta https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-app-types
h
Ah awesome, will try that out. Thanks a lot
When I use a mobile/native app I am actually getting to this line here i.e., the browser says I authenticated correctly. Here are the logs from the admin
Copy code
flyteadmin-858647d86d-mxzv5 flyteadmin {"json":{"src":"handlers.go:209"},"level":"debug","msg":"Found existing metadata Bearer JWT_TOKEN","ts":"2022-09-04T06:42:05Z"}
flyteadmin-858647d86d-mxzv5 flyteadmin {"json":{"src":"handlers.go:237"},"level":"debug","msg":"Running authentication gRPC interceptor","ts":"2022-09-04T06:42:05Z"}
flyteadmin-858647d86d-mxzv5 flyteadmin {"json":{"src":"handlers.go:247"},"level":"info","msg":"Failed to parse Access Token from context. Will attempt to find IDToken. Error: failed to verify id token signature","ts":"2022-09-04T06:42:05Z"}
flyteadmin-858647d86d-mxzv5 flyteadmin {"json":{"src":"token.go:103"},"level":"debug","msg":"Could not retrieve id token from metadata rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken","ts":"2022-09-04T06:42:05Z"}
Here is the problem most likely
Copy code
flyteadmin-858647d86d-mxzv5 flyteadmin {"json":{"src":"handlers.go:247"},"level":"info","msg":"Failed to parse Access Token from context. Will attempt to find IDToken. Error: failed to verify id token signature","ts":"2022-09-04T06:38:59Z"}
After I am getting the successful auth message (i.e., this https://github.com/flyteorg/flyteidl/blob/master/clients/go/admin/pkce/handle_app_call_back.go#L52) in the browser I am seeing these logs in from flytectl
Copy code
Error: rpc error: code = Unauthenticated desc = token parse error [JWT_VERIFICATION_FAILED] Could not retrieve id token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken
{"json":{"src":"main.go:13"},"level":"error","msg":"rpc error: code = Unauthenticated desc = token parse error [JWT_VERIFICATION_FAILED] Could not retrieve id token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken","ts":"2022-09-04T08:42:05+02:00"}
In the token header I am getting a 
nonce
parameter, seems like OpenID Connect nonce and PKCE are not entirely the same auth flow as to where the token validation takes place
p
Ok that one good news . So the auth flow completes and generated the token but the token is rejected by admin while verifying the signature Nonce is part of the header https://github.com/square/go-jose/blob/v2.5.1/shared.go#L181 So i am not sure if that causing an issue though. Unfortunately the same error is printed at two places , we will have to check both https://github.com/coreos/go-oidc/blob/v2.2.1/jwks.go#L125 https://github.com/coreos/go-oidc/blob/v2.2.1/jwks.go#L140 Assuming first time nothing is in the cache so it should fail at the L140 Can you verify that if you open {baseUrl}/{metadataUrl} that you have property called jwks_uri / jwksUri Open the url pointed to jwks_uri / jwksUri. And note the kid property in the keys returned. There could be multiple keys Can you try pasting your saved token and use the access token from it and verify it on jwt.io . Does the signature show its verified over there . Also note the kid in the header. You can get kid also by running this code Also paste the token in here and can you check the KeyID printed https://go.dev/play/p/CR_fPsT36t_S Does it match the KID from jwks_uri data
h
Yeah the KID in the token matches the KID in the jwks_uri. However, the token says invalid signature when I try to validate it
I think this has something to do with it
p
Thanks for checking that. Lot of discussion and many ways folks have tried diff ways to resolve it. Audience and scope configuration can also lead to this looks like . Can you check what is the audience configured on the Azure ad auth server. In case of okta its the flyte-endpoint. Also the scope , may be introduce a new scope for the app detailed here https://github.com/AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet/issues/609#issuecomment-887524901 @Haytham Abuelfutuh can you suggest what else should we try here.
From the comments 
special processing
from what i get means only the resource server (in our case flyteadmin) who is entitled to receive the token can understand the data .
This err which is not printed anywhere may be can give us more clue https://github.com/coreos/go-oidc/blob/8d771559cf6e5111c9b9159810d0e4538e7cdc82/jwks.go#L135 You can try adding a log statement and consuming it in flyteadmin
h
I followed the steps from the github issue comment, and I am actually getting a valid token now, but the admin fails here after finding a bearer token, I remember us having this issue previously but I am not sure how we solved it
Copy code
flyteadmin-b6b9c465c-vgj57 flyteadmin {"json":{"src":"handlers.go:237"},"level":"debug","msg":"Running authentication gRPC interceptor","ts":"2022-09-06T06:15:34Z"}
flyteadmin-b6b9c465c-vgj57 flyteadmin panic: interface conversion: interface {} is string, not []interface {}
flyteadmin-b6b9c465c-vgj57 flyteadmin
flyteadmin-b6b9c465c-vgj57 flyteadmin goroutine 1772 [running]:
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/flyteorg/flyteadmin/auth/authzserver.verifyClaims(0x23975a0|github.com/flyteorg/flyteadmin/auth/authzserver.verifyClaims(0x23975a0>?, 0xc001efb440)
flyteadmin-b6b9c465c-vgj57 flyteadmin 	/go/src/github.com/flyteorg/flyteadmin/auth/authzserver/provider.go:169 +0x7a6
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/flyteorg/flyteadmin/auth/authzserver.ResourceServer.ValidateAccessToken({{0x2988b00|github.com/flyteorg/flyteadmin/auth/authzserver.ResourceServer.ValidateAccessToken({{0x2988b00>, 0xc0001fee70}, {0xc000a691a0, 0x3, 0x3}}, {0x29ace40?, 0xc001d3d410?}, {0xc001bf2600, 0x20}, {0xc00097b507, ...})
flyteadmin-b6b9c465c-vgj57 flyteadmin 	/go/src/github.com/flyteorg/flyteadmin/auth/authzserver/resource_server.go:46 +0x2df
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/flyteorg/flyteadmin/auth.GRPCGetIdentityFromAccessToken({0x29ace40|github.com/flyteorg/flyteadmin/auth.GRPCGetIdentityFromAccessToken({0x29ace40>, 0xc001d3d410}, {0x29b6ea0, 0xc000848120})
flyteadmin-b6b9c465c-vgj57 flyteadmin 	/go/src/github.com/flyteorg/flyteadmin/auth/token.go:93 +0x1e5
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/flyteorg/flyteadmin/auth.GetAuthenticationInterceptor.func1({0x29ace40|github.com/flyteorg/flyteadmin/auth.GetAuthenticationInterceptor.func1({0x29ace40>, 0xc001d3d410})
flyteadmin-b6b9c465c-vgj57 flyteadmin 	/go/src/github.com/flyteorg/flyteadmin/auth/handlers.go:242 +0xaf
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/grpc-ecosystem/go-grpc-middleware/auth.UnaryServerInterceptor.func1({0x29ace40|github.com/grpc-ecosystem/go-grpc-middleware/auth.UnaryServerInterceptor.func1({0x29ace40>, 0xc001d3d410}, {0x2375fe0, 0xc001c7da40}, 0xc001c3e5e0, 0xc001c3e620)
flyteadmin-b6b9c465c-vgj57 flyteadmin 	/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/auth/auth.go:42 +0x93
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x29ace40|github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x29ace40>?, 0xc001d3d410?}, {0x2375fe0?, 0xc001c7da40?})
flyteadmin-b6b9c465c-vgj57 flyteadmin 	/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/chain.go:25 +0x3a
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/flyteorg/flyteadmin/auth.GetAuthenticationCustomMetadataInterceptor.func1({0x29ace40|github.com/flyteorg/flyteadmin/auth.GetAuthenticationCustomMetadataInterceptor.func1({0x29ace40>, 0xc001d3c300}, {0x2375fe0, 0xc001c7da40}, 0x20?, 0xc001c3e640)
flyteadmin-b6b9c465c-vgj57 flyteadmin 	/go/src/github.com/flyteorg/flyteadmin/auth/handlers.go:213 +0x323
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x29ace40|github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x29ace40>?, 0xc001d3c300?}, {0x2375fe0?, 0xc001c7da40?})
flyteadmin-b6b9c465c-vgj57 flyteadmin 	/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/chain.go:25 +0x3a
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).UnaryServerInterceptor.func1({0x29ace40|github.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).UnaryServerInterceptor.func1({0x29ace40>, 0xc001d3c300}, {0x2375fe0, 0xc001c7da40}, 0x7f03ac8576d8?, 0xc001c3e660)
flyteadmin-b6b9c465c-vgj57 flyteadmin 	/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:107 +0x87
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x29ace40|github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x29ace40>?, 0xc001d3c300?}, {0x2375fe0?, 0xc001c7da40?})
flyteadmin-b6b9c465c-vgj57 flyteadmin 	/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/chain.go:25 +0x3a
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1({0x29ace40|github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1({0x29ace40>, 0xc001d3c300}, {0x2375fe0, 0xc001c7da40}, 0xc001a9eaf0?, 0x20f35e0?)
flyteadmin-b6b9c465c-vgj57 flyteadmin 	/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/chain.go:34 +0xbf
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/flyteorg/flyteidl/gen/pb-go/flyteidl/service._AdminService_ListProjects_Handler({0x2444fa0|github.com/flyteorg/flyteidl/gen/pb-go/flyteidl/service._AdminService_ListProjects_Handler({0x2444fa0>?, 0xc001529000}, {0x29ace40, 0xc001d3c300}, 0xc001cfefc0, 0xc000e111d0)
flyteadmin-b6b9c465c-vgj57 flyteadmin 	/go/pkg/mod/github.com/flyteorg/flyteidl@v1.1.5/gen/pb-go/flyteidl/service/admin.pb.go:1576 +0x138
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://google.golang.org/grpc.(*Server).processUnaryRPC(0xc000d1b180|google.golang.org/grpc.(*Server).processUnaryRPC(0xc000d1b180>, {0x29b5708, 0xc001a7f1e0}, 0xc000a5b0e0, 0xc000a4d200, 0x3c66e58, 0x0)
flyteadmin-b6b9c465c-vgj57 flyteadmin 	/go/pkg/mod/google.golang.org/grpc@v1.46.0/server.go:1283 +0xcfd
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://google.golang.org/grpc.(*Server).handleStream(0xc000d1b180|google.golang.org/grpc.(*Server).handleStream(0xc000d1b180>, {0x29b5708, 0xc001a7f1e0}, 0xc000a5b0e0, 0x0)
flyteadmin-b6b9c465c-vgj57 flyteadmin 	/go/pkg/mod/google.golang.org/grpc@v1.46.0/server.go:1620 +0xa1b
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://google.golang.org/grpc.(*Server).serveStreams.func1.2()|google.golang.org/grpc.(*Server).serveStreams.func1.2()>
flyteadmin-b6b9c465c-vgj57 flyteadmin 	/go/pkg/mod/google.golang.org/grpc@v1.46.0/server.go:922 +0x98
flyteadmin-b6b9c465c-vgj57 flyteadmin created by <http://google.golang.org/grpc.(*Server).serveStreams.func1|google.golang.org/grpc.(*Server).serveStreams.func1>
flyteadmin-b6b9c465c-vgj57 flyteadmin 	/go/pkg/mod/google.golang.org/grpc@v1.46.0/server.go:920 +0x28a
I updated my 
allowedAudience
to be as I was getting a wrong audience error (to match the audience in the token I am getting for flytectl)
Copy code
allowedAudience:
  - consoleClientId
  - propellerClientId
  - <api://flytectlClientId>
p
scp is array of strings but seems in your case it returns a string when it parses the azure ad token. https://datatracker.ietf.org/doc/html/draft-ietf-oauth-token-exchange-06#section-4.2 can you check if this can be fixed in azure ad configuration or may be you can try adding a check in flyteadmin code if it can’t be typecasted to slice of strings then just populate it as single element scp list
h
yeah the claims are actually just a single string in this case
p
Oh i think we just need a change here https://github.com/flyteorg/flyteadmin/blob/master/auth/authzserver/provider.go#L169 . Is that right since only the scopesClaim is not returning a list, though the key ScopeClaim exists
h
Yes the 
scp
key exists, however it’s only a string instead of list of strings
p
Created a image with the check ghcr.io/flyteorg/flyteadmin:v1.1.37-scp-fix . Do you want to try it . Needs to replaced in flyteadmin serve command image
h
Yes, I will try tonight as some teams are currently doing experiments on the Platform
Thanks a lot Prafulla
Here right,
Copy code
containers:
      - command:
        - flyteadmin
        - --config
        - /etc/flyte/config/*.yaml
        - serve
        image: <http://cr.flyte.org/flyteorg/flyteadmin-release:v1.1.0|cr.flyte.org/flyteorg/flyteadmin-release:v1.1.0>
        imagePullPolicy: IfNotPresent
p
yes
h
Admin is not failing anymore, but I am getting this error now, I think it is due to that the scopes that I am requesting is not matching the scopes in the JWT. Where is this checked?
Copy code
☁  ~  flytectl get projects
{"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [storage] updated. No update handler registered.","ts":"2022-09-07T06:59:26+02:00"}
{"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [root] updated. No update handler registered.","ts":"2022-09-07T06:59:26+02:00"}
{"json":{"src":"viper.go:400"},"level":"debug","msg":"Config section [admin] updated. Firing updated event.","ts":"2022-09-07T06:59:26+02:00"}
{"json":{"src":"auth_flow_orchestrator.go:37"},"level":"debug","msg":"got a response from the refresh grant for old expiry 2022-09-07 08:23:35.302445 +0200 CEST with new expiry 2022-09-07 08:23:35.302445 +0200 CEST","ts":"2022-09-07T06:59:26+02:00"}
{"json":{"src":"client.go:64"},"level":"info","msg":"Initialized Admin client","ts":"2022-09-07T06:59:26+02:00"}
Error: rpc error: code = Unauthenticated desc = authenticated user doesn't have required scope
{"json":{"src":"main.go:13"},"level":"error","msg":"rpc error: code = Unauthenticated desc = authenticated user doesn't have required scope","ts":"2022-09-07T06:59:26+02:00"}
p
A scope named all is required in your authorization server https://docs.flyte.org/en/latest/deployment/cluster_config/auth_setup.html#oauth2-authorization-server
Copy code
Under Scopes, click Add Scope. Set the name to all (required) and check Require user consent for this scope (recommended).
Flyteadmin has a hard expectation to have this scope . https://github.com/flyteorg/flyteadmin/blob/master/auth/interceptor.go#L19
You also would need to add that scope in here
Copy code
thirdPartyConfig:
            flyteClient:
                # 3. Replace with a new Native/Public Client ID provisioned in the custom authorization server.
                clientId: flytectl

                # This should not change
                redirectUri: <http://localhost:53593/callback>

                # 4. "all" is a required scope and must be configured in the custom authorization server.
                scopes:
                - offline
                - all
h
That works! Awesome
If I don’t have a cookie already in my browser I get a timeout after like 20-30 seconds. Looks like the duration is set to two minutes in the pkce config https://github.com/flyteorg/flyteidl/blob/a93ef49d24aaac0b331035af0b15c76e16b4f4f4/clients/go/admin/config.go#L93
But with a cookie already present it works like a charm
p
Awesome . Finally we got this working .
When you mean timeout after 20-30 secs , does the session timeout for you . because the new default is 2 minutes, but yeah thats not yet consumed in fllytectl . Just got merged a few hours ago https://github.com/flyteorg/flytectl/pull/350 The earlier default was 30 secs which is what you might be experiencing
You can also pass this from commandline 
--admin.pkceConfig.timeout 2m
A lot of hurdles before we got this running.thanks for sticking through the entire journey as this is flytes first azure ad setup and we learnt a lot of things along the way both for Azure ad and flyte auth. Would be great if you can help with documentation on this that can help the community. Thanks again. Cc: @Haytham Abuelfutuh
h
Copy code
flytectl get projects --admin.pkceConfig.timeout 2m
That works!
Yes, I will make sure to write some proper docs. Quite busy with other stuff at work at the moment, but I will try to get started this week. I think in general it just boils down to poor docs from Azure side on how to configure the apps for the different use cases
Also, many thanks for the assistance from Flyte team. Really awesome 🙂
p
Sounds good. Thanks Hampus
203 Views
Open in Slack
PreviousNext
Powered by