melodic-angle-13491
05/22/2023, 7:38 PMError: Connection Info: [Endpoint: dns:///flyte.localhost:80, InsecureConnection?: true, AuthMode: ClientSecret]: rpc error: code = Unauthenticated desc = authenticated user doesn't have required scope
LinkerD confirms that it's the flyteadmin container that responds with Unauthenticated.
This is quite perplexing and I've verified that the token is sent and it contains all the scopes required: "scope": "all email offline profile"
I hope people can give me some hints as I've been bashing my head for way to long against this. I'm just trying to have a flytectl register stuff in CI.melodic-angle-13491
05/22/2023, 8:31 PMflyte-authorization
header:
{
"exp": 1684787333,
"iat": 1684787033,
"jti": "984df155-c8fc-41f4-9e33-65e38eb51e06",
"iss": "<http://keycloak.localhost/realms/flyte>",
"aud": "account",
"sub": "02bf7fe8-1b4a-4b5f-a08c-378eb26be405",
"typ": "Bearer",
"azp": "flytectl",
"session_state": "43678964-d918-4557-b9ae-ce3639e8cfe3",
"acr": "1",
"allowed-origins": [
"/*"
],
"realm_access": {
"roles": [
"default-roles-flyte",
"offline_access",
"uma_authorization"
]
},
"resource_access": {
"account": {
"roles": [
"manage-account",
"manage-account-links",
"view-profile"
]
}
},
"scope": "openid profile email all offline",
"sid": "43678964-d918-4557-b9ae-ce3639e8cfe3",
"email_verified": false,
"clientHost": "10.42.0.50",
"preferred_username": "service-account-flytectl",
"clientAddress": "10.42.0.50",
"client_id": "flytectl"
}
melodic-angle-13491
05/22/2023, 8:32 PM{
"exp": 1684787117,
"iat": 1684786817,
"auth_time": 0,
"jti": "07555916-cb38-485f-ae29-e4af0a183dfb",
"iss": "<http://keycloak.localhost/realms/flyte>",
"aud": "flytectl",
"sub": "02bf7fe8-1b4a-4b5f-a08c-378eb26be405",
"typ": "ID",
"azp": "flytectl",
"session_state": "f6c36f22-0614-4e58-811f-0ceb02916037",
"at_hash": "BqKQDMlf3921-RZSmzaHtA",
"acr": "1",
"sid": "f6c36f22-0614-4e58-811f-0ceb02916037",
"email_verified": false,
"clientHost": "10.42.0.50",
"preferred_username": "service-account-flytectl",
"clientAddress": "10.42.0.50",
"client_id": "flytectl"
}
melodic-angle-13491
05/22/2023, 8:34 PMtall-lock-23197
insecure
to true in your flytectl config?melodic-angle-13491
05/23/2023, 6:38 AMtall-lock-23197
melodic-angle-13491
05/23/2023, 8:49 AMmelodic-angle-13491
05/23/2023, 8:51 AMmelodic-angle-13491
05/23/2023, 8:52 AMtall-lock-23197
melodic-angle-13491
05/23/2023, 9:35 AMmelodic-angle-13491
05/23/2023, 9:35 AMmelodic-angle-13491
05/23/2023, 10:07 AMflyte-authentication
header contains Bearer <token>
which, when decoded, contains the proper scopes.melodic-angle-13491
05/23/2023, 10:08 AMfreezing-airport-6809
melodic-angle-13491
05/23/2023, 1:04 PMfreezing-airport-6809
melodic-angle-13491
05/23/2023, 1:26 PMfreezing-airport-6809
melodic-angle-13491
05/23/2023, 1:30 PMfreezing-airport-6809
melodic-angle-13491
05/23/2023, 2:06 PMWhy are you trying to run auth in sandbox?
I'm not running the sandbox setup as described in the docs, but the single cluster production setup. This is an exploratory experiment to find all the component behaviors and failure modes before shaping it into more hardened environments.
You agree that in real setting you will have TLS and only then enable authYes but only for user auth probably, not service account based activity like
flytectl register
as these processes most likely will run in the same cluster and will be covered by mTLS and network policies.freezing-airport-6809
melodic-angle-13491
05/23/2023, 2:24 PMfreezing-airport-6809
icy-agent-73298
05/23/2023, 4:59 PMmelodic-angle-13491
05/23/2023, 5:05 PMClientSecret
as a method. insecure: true
as mentioned before. Both the all
and offline
scopes are also set but are optional as they get included by default by keycloak.
Flyteadmin has been configured with useAuth
, external auth server, user auth set to a separate oauth2 client that can only do pkce, and the third party set to the same flytectl client as previously mentioned.icy-agent-73298
05/23/2023, 5:29 PM"scope": "openid profile email all offline",
Whereas we expect it to be
scp: …..
https://github.com/flyteorg/flyteadmin/blob/master/auth/authzserver/provider.go#L38
We can add a change to support this but most of the providers that the community has deployed have not run into this particular AFAIK but i might be wrong. I can provide a sandbox build of flyteadmin for you to try and see if that works for you and if this something configurable in keycloak to use scp to send the scope claims then that would work toomelodic-angle-13491
05/23/2023, 5:33 PMicy-agent-73298
05/23/2023, 5:39 PMicy-agent-73298
05/23/2023, 5:41 PMmelodic-angle-13491
05/23/2023, 5:46 PMicy-agent-73298
05/23/2023, 5:50 PMicy-agent-73298
05/23/2023, 5:51 PMmelodic-angle-13491
05/23/2023, 5:55 PMicy-agent-73298
05/23/2023, 5:59 PMmelodic-angle-13491
05/23/2023, 6:02 PMicy-agent-73298
05/23/2023, 6:04 PMmelodic-angle-13491
05/26/2023, 1:17 PMmelodic-angle-13491
05/26/2023, 1:18 PMmelodic-angle-13491
05/26/2023, 1:18 PMicy-agent-73298
05/30/2023, 9:48 PMmelodic-angle-13491
05/31/2023, 8:36 PMscp
claim and the docs don't mention you needing to configure the IDP to include that claim.icy-agent-73298
05/31/2023, 9:35 PMmelodic-angle-13491
05/31/2023, 9:52 PMicy-agent-73298
05/31/2023, 11:00 PM