high-park-82026
high-park-82026
helpful-crowd-74546
08/29/2022, 5:25 PM~ k get configmap flyte-{propeller, scheduler}-config
admin.yaml: |
admin:
clientId: clientId
clientSecretLocation: /etc/secrets/client_secret
endpoint: flyteadmin:81
insecure: true
icy-agent-73298
08/29/2022, 5:25 PMicy-agent-73298
08/29/2022, 5:26 PMhelpful-crowd-74546
08/29/2022, 5:27 PMadmin:
endpoint: flyteadmin:81
insecure: true
clientId: "{{ .Values.secrets.adminOauthClientCredentials.clientId }}"
clientSecretLocation: /etc/secrets/client_secret
scopes: ["myScopes"]
icy-agent-73298
08/29/2022, 5:28 PMadmin:
endpoint: flyteadmin:81
insecure: true
clientId: "{{ .Values.secrets.adminOauthClientCredentials.clientId }}"
clientSecretLocation: /etc/secrets/client_secret
scopes:
- myscopes
helpful-crowd-74546
08/29/2022, 5:29 PMicy-agent-73298
08/29/2022, 5:29 PMhelpful-crowd-74546
08/29/2022, 5:37 PMCool! Your authentication was successful and you can close the window.
But I am getting this response now from the admin
Error: rpc error: code = Unauthenticated desc = token parse error [JWT_VERIFICATION_FAILED] Could not retrieve id token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken
helpful-crowd-74546
08/29/2022, 5:38 PMhelpful-crowd-74546
08/29/2022, 5:57 PMhigh-park-82026
helpful-crowd-74546
08/29/2022, 7:10 PMhelpful-crowd-74546
08/29/2022, 7:15 PMhigh-park-82026
icy-agent-73298
08/30/2022, 5:59 AMhelpful-crowd-74546
08/30/2022, 6:18 AMhelpful-crowd-74546
08/30/2022, 6:19 AMappAuth:
authServerType: External
externalAuthServer:
baseUrl: <https://login.microsoftonline.com/tenant/v2.0>
metadataUrl: .well-known/openid-configuration
allowedAudience:
- clientid
This is what I am doing for the client credentials flow, and I am getting invalid audience error for the PKCE flow nowhelpful-crowd-74546
08/30/2022, 6:20 AMicy-agent-73298
08/30/2022, 6:22 AMhelpful-crowd-74546
08/30/2022, 6:26 AMhelpful-crowd-74546
08/30/2022, 6:26 AMhelpful-crowd-74546
08/30/2022, 6:28 AMicy-agent-73298
08/30/2022, 6:28 AMicy-agent-73298
08/30/2022, 6:28 AMicy-agent-73298
08/30/2022, 6:30 AMhelpful-crowd-74546
08/30/2022, 6:32 AMflyteadmin-76f6476884-9vnxq flyteadmin panic: interface conversion: interface {} is string, not []interface {}
flyteadmin-76f6476884-9vnxq flyteadmin
helpful-crowd-74546
08/30/2022, 6:32 AMallowedAudience:
- propellerClientId
- flytectlClientId
icy-agent-73298
08/30/2022, 6:37 AMkey1: [value1,value2,value3,value4,value5]
icy-agent-73298
08/30/2022, 6:38 AMhelpful-crowd-74546
08/30/2022, 6:46 AMicy-agent-73298
08/30/2022, 6:48 AMhelpful-crowd-74546
08/31/2022, 8:11 AMTokens issued for the 'Single-Page Application' client-type may only be redeemed via cross-origin requests
which seems to be discussed in this thread e.g., this solution https://stackoverflow.com/a/67280275. Is this something I can add to the request?icy-agent-73298
08/31/2022, 5:19 PMhelpful-crowd-74546
08/31/2022, 6:29 PM{"json":{"src":"token_source_provider.go:121"},"level":"error","msg":"Error fetching token using auth flow due to error while exchanging auth code due to oauth2: cannot fetch token: 400 Bad Request\nResponse: {\"error\":\"invalid_request\",\"error_description\":\": Tokens issued for the 'Single-Page Application' client-type may only be redeemed via cross-origin requests
helpful-crowd-74546
08/31/2022, 6:35 PMserveMux
icy-agent-73298
09/01/2022, 7:23 AMhelpful-crowd-74546
09/01/2022, 2:12 PMserver.yaml
configMap:
server:
grpcPort: 8089
httpPort: 8088
security:
allowCors: true
allowedHeaders:
- Content-Type
allowedOrigins:
- '*'
secure: false
useAuth: true
helpful-crowd-74546
09/01/2022, 2:14 PM{"json":{"src":"token_source_provider.go:121"},"level":"error","msg":"Error fetching token using auth flow due to error while exchanging auth code due to oauth2: cannot fetch token:
icy-agent-73298
09/01/2022, 2:21 PMhelpful-crowd-74546
09/01/2022, 2:23 PMflyteadmin-858647d86d-8f4h8 flyteadmin {"json":{"src":"handlers.go:237"},"level":"debug","msg":"Running authentication gRPC interceptor","ts":"2022-09-01T14:23:10Z"}
flyteadmin-858647d86d-8f4h8 flyteadmin {"json":{"src":"token.go:83"},"level":"debug","msg":"Could not retrieve bearer token from metadata rpc error: code = Unauthenticated desc = Request unauthenticated with Bearer","ts":"2022-09-01T14:23:10Z"}
flyteadmin-858647d86d-8f4h8 flyteadmin {"json":{"src":"handlers.go:247"},"level":"info","msg":"Failed to parse Access Token from context. Will attempt to find IDToken. Error: [JWT_VERIFICATION_FAILED] Could not retrieve bearer token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with Bearer","ts":"2022-09-01T14:23:10Z"}
flyteadmin-858647d86d-8f4h8 flyteadmin {"json":{"src":"token.go:103"},"level":"debug","msg":"Could not retrieve id token from metadata rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken","ts":"2022-09-01T14:23:10Z"}
icy-agent-73298
09/01/2022, 2:23 PMTokens issued for the 'Single-Page Application' client-type may only be redeemed via cross-origin requests
helpful-crowd-74546
09/01/2022, 2:24 PM{"json":{"src":"token_source_provider.go:121"},"level":"error","msg":"Error fetching token using auth flow due to error while exchanging auth code due to oauth2: cannot fetch token: 400 Bad Request\nResponse: {\"error\":\"invalid_request\",\"error_description\":\"AADSTS9002327: Tokens issued for the 'Single-Page Application' client-type may only be redeemed via cross-origin requests'..}
{"json":{"src":"client.go:188"},"level":"warning","msg":"Starting an unauthenticated client because: error while exchanging auth code due to oauth2: cannot fetch token: 400 Bad Request\nResponse: {\"error\":\"invalid_request\",\"error_description\":\"AADSTS9002327: Tokens issued for the 'Single-Page Application' client-type may only be redeemed via cross-origin requests"...}
{"json":{"src":"client.go:64"},"level":"info","msg":"Initialized Admin client","ts":"2022-09-01T16:23:09+02:00"}
Error: rpc error: code = Unauthenticated desc = token parse error [JWT_VERIFICATION_FAILED] Could not retrieve id token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken
{"json":{"src":"main.go:13"},"level":"error","msg":"rpc error: code = Unauthenticated desc = token parse error [JWT_VERIFICATION_FAILED] Could not retrieve id token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken","ts":"2022-09-01T16:23:10+02:00"}
helpful-crowd-74546
09/01/2022, 2:25 PMhelpful-crowd-74546
09/01/2022, 2:26 PMicy-agent-73298
09/02/2022, 6:32 AMicy-agent-73298
09/02/2022, 6:46 AMhelpful-crowd-74546
09/03/2022, 7:12 AMhelpful-crowd-74546
09/04/2022, 6:44 AMflyteadmin-858647d86d-mxzv5 flyteadmin {"json":{"src":"handlers.go:209"},"level":"debug","msg":"Found existing metadata Bearer JWT_TOKEN","ts":"2022-09-04T06:42:05Z"}
flyteadmin-858647d86d-mxzv5 flyteadmin {"json":{"src":"handlers.go:237"},"level":"debug","msg":"Running authentication gRPC interceptor","ts":"2022-09-04T06:42:05Z"}
flyteadmin-858647d86d-mxzv5 flyteadmin {"json":{"src":"handlers.go:247"},"level":"info","msg":"Failed to parse Access Token from context. Will attempt to find IDToken. Error: failed to verify id token signature","ts":"2022-09-04T06:42:05Z"}
flyteadmin-858647d86d-mxzv5 flyteadmin {"json":{"src":"token.go:103"},"level":"debug","msg":"Could not retrieve id token from metadata rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken","ts":"2022-09-04T06:42:05Z"}
Here is the problem most likely
flyteadmin-858647d86d-mxzv5 flyteadmin {"json":{"src":"handlers.go:247"},"level":"info","msg":"Failed to parse Access Token from context. Will attempt to find IDToken. Error: failed to verify id token signature","ts":"2022-09-04T06:38:59Z"}
After I am getting the successful auth message (i.e., this https://github.com/flyteorg/flyteidl/blob/master/clients/go/admin/pkce/handle_app_call_back.go#L52) in the browser I am seeing these logs in from flytectl
Error: rpc error: code = Unauthenticated desc = token parse error [JWT_VERIFICATION_FAILED] Could not retrieve id token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken
{"json":{"src":"main.go:13"},"level":"error","msg":"rpc error: code = Unauthenticated desc = token parse error [JWT_VERIFICATION_FAILED] Could not retrieve id token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken","ts":"2022-09-04T08:42:05+02:00"}
helpful-crowd-74546
09/04/2022, 7:10 AMnonce
parameter, seems like OpenID Connect nonce and PKCE are not entirely the same auth flow as to where the token validation takes placeicy-agent-73298
09/05/2022, 7:09 AMhelpful-crowd-74546
09/05/2022, 11:28 AMhelpful-crowd-74546
09/05/2022, 11:29 AMhelpful-crowd-74546
09/05/2022, 11:30 AMicy-agent-73298
09/05/2022, 11:52 AMicy-agent-73298
09/05/2022, 11:56 AMspecial processing
from what i get means only the resource server (in our case flyteadmin) who is entitled to receive the token can understand the data .icy-agent-73298
09/05/2022, 12:02 PMhelpful-crowd-74546
09/06/2022, 6:21 AMflyteadmin-b6b9c465c-vgj57 flyteadmin {"json":{"src":"handlers.go:237"},"level":"debug","msg":"Running authentication gRPC interceptor","ts":"2022-09-06T06:15:34Z"}
flyteadmin-b6b9c465c-vgj57 flyteadmin panic: interface conversion: interface {} is string, not []interface {}
flyteadmin-b6b9c465c-vgj57 flyteadmin
flyteadmin-b6b9c465c-vgj57 flyteadmin goroutine 1772 [running]:
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/flyteorg/flyteadmin/auth/authzserver.verifyClaims(0x23975a0|github.com/flyteorg/flyteadmin/auth/authzserver.verifyClaims(0x23975a0>?, 0xc001efb440)
flyteadmin-b6b9c465c-vgj57 flyteadmin /go/src/github.com/flyteorg/flyteadmin/auth/authzserver/provider.go:169 +0x7a6
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/flyteorg/flyteadmin/auth/authzserver.ResourceServer.ValidateAccessToken({{0x2988b00|github.com/flyteorg/flyteadmin/auth/authzserver.ResourceServer.ValidateAccessToken({{0x2988b00>, 0xc0001fee70}, {0xc000a691a0, 0x3, 0x3}}, {0x29ace40?, 0xc001d3d410?}, {0xc001bf2600, 0x20}, {0xc00097b507, ...})
flyteadmin-b6b9c465c-vgj57 flyteadmin /go/src/github.com/flyteorg/flyteadmin/auth/authzserver/resource_server.go:46 +0x2df
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/flyteorg/flyteadmin/auth.GRPCGetIdentityFromAccessToken({0x29ace40|github.com/flyteorg/flyteadmin/auth.GRPCGetIdentityFromAccessToken({0x29ace40>, 0xc001d3d410}, {0x29b6ea0, 0xc000848120})
flyteadmin-b6b9c465c-vgj57 flyteadmin /go/src/github.com/flyteorg/flyteadmin/auth/token.go:93 +0x1e5
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/flyteorg/flyteadmin/auth.GetAuthenticationInterceptor.func1({0x29ace40|github.com/flyteorg/flyteadmin/auth.GetAuthenticationInterceptor.func1({0x29ace40>, 0xc001d3d410})
flyteadmin-b6b9c465c-vgj57 flyteadmin /go/src/github.com/flyteorg/flyteadmin/auth/handlers.go:242 +0xaf
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/grpc-ecosystem/go-grpc-middleware/auth.UnaryServerInterceptor.func1({0x29ace40|github.com/grpc-ecosystem/go-grpc-middleware/auth.UnaryServerInterceptor.func1({0x29ace40>, 0xc001d3d410}, {0x2375fe0, 0xc001c7da40}, 0xc001c3e5e0, 0xc001c3e620)
flyteadmin-b6b9c465c-vgj57 flyteadmin /go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/auth/auth.go:42 +0x93
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x29ace40|github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x29ace40>?, 0xc001d3d410?}, {0x2375fe0?, 0xc001c7da40?})
flyteadmin-b6b9c465c-vgj57 flyteadmin /go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/chain.go:25 +0x3a
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/flyteorg/flyteadmin/auth.GetAuthenticationCustomMetadataInterceptor.func1({0x29ace40|github.com/flyteorg/flyteadmin/auth.GetAuthenticationCustomMetadataInterceptor.func1({0x29ace40>, 0xc001d3c300}, {0x2375fe0, 0xc001c7da40}, 0x20?, 0xc001c3e640)
flyteadmin-b6b9c465c-vgj57 flyteadmin /go/src/github.com/flyteorg/flyteadmin/auth/handlers.go:213 +0x323
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x29ace40|github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x29ace40>?, 0xc001d3c300?}, {0x2375fe0?, 0xc001c7da40?})
flyteadmin-b6b9c465c-vgj57 flyteadmin /go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/chain.go:25 +0x3a
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).UnaryServerInterceptor.func1({0x29ace40|github.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).UnaryServerInterceptor.func1({0x29ace40>, 0xc001d3c300}, {0x2375fe0, 0xc001c7da40}, 0x7f03ac8576d8?, 0xc001c3e660)
flyteadmin-b6b9c465c-vgj57 flyteadmin /go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:107 +0x87
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x29ace40|github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x29ace40>?, 0xc001d3c300?}, {0x2375fe0?, 0xc001c7da40?})
flyteadmin-b6b9c465c-vgj57 flyteadmin /go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/chain.go:25 +0x3a
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1({0x29ace40|github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1({0x29ace40>, 0xc001d3c300}, {0x2375fe0, 0xc001c7da40}, 0xc001a9eaf0?, 0x20f35e0?)
flyteadmin-b6b9c465c-vgj57 flyteadmin /go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/chain.go:34 +0xbf
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://github.com/flyteorg/flyteidl/gen/pb-go/flyteidl/service._AdminService_ListProjects_Handler({0x2444fa0|github.com/flyteorg/flyteidl/gen/pb-go/flyteidl/service._AdminService_ListProjects_Handler({0x2444fa0>?, 0xc001529000}, {0x29ace40, 0xc001d3c300}, 0xc001cfefc0, 0xc000e111d0)
flyteadmin-b6b9c465c-vgj57 flyteadmin /go/pkg/mod/github.com/flyteorg/flyteidl@v1.1.5/gen/pb-go/flyteidl/service/admin.pb.go:1576 +0x138
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://google.golang.org/grpc.(*Server).processUnaryRPC(0xc000d1b180|google.golang.org/grpc.(*Server).processUnaryRPC(0xc000d1b180>, {0x29b5708, 0xc001a7f1e0}, 0xc000a5b0e0, 0xc000a4d200, 0x3c66e58, 0x0)
flyteadmin-b6b9c465c-vgj57 flyteadmin /go/pkg/mod/google.golang.org/grpc@v1.46.0/server.go:1283 +0xcfd
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://google.golang.org/grpc.(*Server).handleStream(0xc000d1b180|google.golang.org/grpc.(*Server).handleStream(0xc000d1b180>, {0x29b5708, 0xc001a7f1e0}, 0xc000a5b0e0, 0x0)
flyteadmin-b6b9c465c-vgj57 flyteadmin /go/pkg/mod/google.golang.org/grpc@v1.46.0/server.go:1620 +0xa1b
flyteadmin-b6b9c465c-vgj57 flyteadmin <http://google.golang.org/grpc.(*Server).serveStreams.func1.2()|google.golang.org/grpc.(*Server).serveStreams.func1.2()>
flyteadmin-b6b9c465c-vgj57 flyteadmin /go/pkg/mod/google.golang.org/grpc@v1.46.0/server.go:922 +0x98
flyteadmin-b6b9c465c-vgj57 flyteadmin created by <http://google.golang.org/grpc.(*Server).serveStreams.func1|google.golang.org/grpc.(*Server).serveStreams.func1>
flyteadmin-b6b9c465c-vgj57 flyteadmin /go/pkg/mod/google.golang.org/grpc@v1.46.0/server.go:920 +0x28a
helpful-crowd-74546
09/06/2022, 6:22 AMallowedAudience
to be as I was getting a wrong audience error (to match the audience in the token I am getting for flytectl)
allowedAudience:
- consoleClientId
- propellerClientId
- <api://flytectlClientId>
icy-agent-73298
09/06/2022, 6:43 AMhelpful-crowd-74546
09/06/2022, 9:27 AMhelpful-crowd-74546
09/06/2022, 9:29 AMicy-agent-73298
09/06/2022, 10:24 AMhelpful-crowd-74546
09/06/2022, 11:04 AMscp
key exists, however it’s only a string instead of list of stringsicy-agent-73298
09/06/2022, 11:25 AMhelpful-crowd-74546
09/06/2022, 11:35 AMhelpful-crowd-74546
09/06/2022, 11:35 AMhelpful-crowd-74546
09/06/2022, 11:39 AMcontainers:
- command:
- flyteadmin
- --config
- /etc/flyte/config/*.yaml
- serve
image: <http://cr.flyte.org/flyteorg/flyteadmin-release:v1.1.0|cr.flyte.org/flyteorg/flyteadmin-release:v1.1.0>
imagePullPolicy: IfNotPresent
icy-agent-73298
09/06/2022, 11:39 AMhelpful-crowd-74546
09/07/2022, 5:02 AM☁ ~ flytectl get projects
{"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [storage] updated. No update handler registered.","ts":"2022-09-07T06:59:26+02:00"}
{"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [root] updated. No update handler registered.","ts":"2022-09-07T06:59:26+02:00"}
{"json":{"src":"viper.go:400"},"level":"debug","msg":"Config section [admin] updated. Firing updated event.","ts":"2022-09-07T06:59:26+02:00"}
{"json":{"src":"auth_flow_orchestrator.go:37"},"level":"debug","msg":"got a response from the refresh grant for old expiry 2022-09-07 08:23:35.302445 +0200 CEST with new expiry 2022-09-07 08:23:35.302445 +0200 CEST","ts":"2022-09-07T06:59:26+02:00"}
{"json":{"src":"client.go:64"},"level":"info","msg":"Initialized Admin client","ts":"2022-09-07T06:59:26+02:00"}
Error: rpc error: code = Unauthenticated desc = authenticated user doesn't have required scope
{"json":{"src":"main.go:13"},"level":"error","msg":"rpc error: code = Unauthenticated desc = authenticated user doesn't have required scope","ts":"2022-09-07T06:59:26+02:00"}
icy-agent-73298
09/07/2022, 5:27 AMUnder Scopes, click Add Scope. Set the name to all (required) and check Require user consent for this scope (recommended).
icy-agent-73298
09/07/2022, 5:28 AMicy-agent-73298
09/07/2022, 5:42 AMthirdPartyConfig:
flyteClient:
# 3. Replace with a new Native/Public Client ID provisioned in the custom authorization server.
clientId: flytectl
# This should not change
redirectUri: <http://localhost:53593/callback>
# 4. "all" is a required scope and must be configured in the custom authorization server.
scopes:
- offline
- all
helpful-crowd-74546
09/07/2022, 8:08 AMhelpful-crowd-74546
09/07/2022, 8:09 AMhelpful-crowd-74546
09/07/2022, 8:09 AMicy-agent-73298
09/07/2022, 8:43 AMicy-agent-73298
09/07/2022, 8:44 AMicy-agent-73298
09/07/2022, 8:47 AM--admin.pkceConfig.timeout 2m
icy-agent-73298
09/07/2022, 8:59 AMhelpful-crowd-74546
09/07/2022, 9:10 AMflytectl get projects --admin.pkceConfig.timeout 2m
helpful-crowd-74546
09/07/2022, 9:10 AMhelpful-crowd-74546
09/07/2022, 9:11 AMhelpful-crowd-74546
09/07/2022, 9:25 AMicy-agent-73298
09/07/2022, 9:38 AM