Cornelis Boon
03/22/2024, 3:03 PMadmin.endpoint
to dns:///user:pass@domain
-> could not not contact DNS servers (perhaps I need to mveo the username/pass into some header. Unclear how to do this via config file)
โข set admin.endpoint
to <http://user:pass@domain>
-> invalid ipv6 url
โข set admin.endpoint
to <dns://domain>
-> Trying to connect an http1.x serverCornelis Boon
03/22/2024, 3:41 PMadmin.endpoint
to <dns://domain:8089>
and admin.authorizationHeader
to Basic BASE64(user:pass)
. Now getting:
Failed to connect to remote host: FD Shutdown
Also getting this if I don't set the authHeader, so not sure. ๐คCornelis Boon
03/22/2024, 3:45 PMFailed to dial target host "domain:8089": context deadline exceeded
Both with and without the auth.
HTTP ingress seems to work fine btw. Can reach console page via domain/console
Cornelis Boon
03/22/2024, 3:46 PMCornelis Boon
03/22/2024, 4:10 PMYee
Yee
Yee
Cornelis Boon
03/22/2024, 4:12 PMYee
FAKE_SECRET_NAME
env varCornelis Boon
03/22/2024, 4:13 PMYee
Cornelis Boon
03/22/2024, 4:15 PMCornelis Boon
03/22/2024, 4:17 PMCornelis Boon
03/24/2024, 12:15 PMpyflyte run remote-task list
for example. When I try to do this programatically by using FlyteRemote
, I also get 401 when I try to fetch the workflow/task.
Then I dug deeper, and it seems to be an issue with how the client sets itself up.
If I do
e.g.
remote = FlyteRemote(config=Config.auto(), default_project="flytesnacks",default_domain="development")
flyte_wf = remote.fetch_workflow(name="<http://workflows.example.wf|workflows.example.wf>")
I get the same issue, but with a stacktrace (in ๐งต ). It seems like it's trying to make an unauthenticated request to an AuthMetadataService
to get some config before actually making authenticated requests.
But since that service is likely also behind the basic-auth ingress, that request never makes it through.
Does this mean basic-auth as I've setup is currently not supported or am I missing something?\Cornelis Boon
03/24/2024, 12:17 PMโญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ Traceback (most recent call last) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ in <module>:1 โ
โ โ
โ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/flytekit/remote/remote.py:378 in fetch_workflow โ
โ โ
โ โฑ 378 โ โ โ self.client.list_workflows_paginated, โ
โ โ
โ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/flytekit/remote/remote.py:236 in client โ
โ โ
โ โฑ 236 โ โ โ self._client = SynchronousFlyteClient(self.config.platform, **self._kwargs) โ
โ โ
โ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/flytekit/clients/raw.py:50 in __init__ โ
โ โ
โ โฑ 50 โ โ โ cfg, upgrade_channel_to_authenticated(cfg, upgrade_channel_to_proxy_authenti โ
โ โ
โ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/flytekit/clients/auth_helper.py:140 in upgrade_channel_to_authenticated โ
โ โ
โ โฑ 140 โ authenticator = get_authenticator(cfg, RemoteClientConfigStore(in_channel)) โ
โ โ
โ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/flytekit/clients/auth_helper.py:75 in get_authenticator โ
โ โ
โ โฑ 75 โ โ return ClientCredentialsAuthenticator( โ
โ โ
โ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/flytekit/clients/auth/authenticator.py:213 in __init__ โ
โ โ
โ โฑ 213 โ โ cfg = cfg_store.get_client_config() โ
โ โ
โ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/flytekit/clients/auth_helper.py:38 in get_client_config โ
โ โ
โ โฑ 38 โ โ public_client_config = metadata_service.GetPublicClientConfig(PublicClientAuthCo โ
โ โ
โ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/grpc/_interceptor.py:277 in __call__ โ
โ โ
โ โฑ 277 โ โ response, ignored_call = self._with_call( โ
โ โ
โ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/grpc/_interceptor.py:332 in _with_call โ
โ โ
โ โฑ 332 โ โ return call.result(), call โ
โ โ
โ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/grpc/_channel.py:439 in result โ
โ โ
โ โฑ 439 โ โ raise self โ
โ โ
โ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/grpc/_interceptor.py:315 in continuation โ
โ โ
โ โฑ 315 โ โ โ โ response, call = self._thunk(new_method).with_call( โ
โ โ
โ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/grpc/_channel.py:1193 in with_call โ
โ โ
โ โฑ 1193 โ โ return _end_unary_response_blocking(state, call, True, None) โ
โ โ
โ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/grpc/_channel.py:1005 in _end_unary_response_blocking โ
โ โ
โ โฑ 1005 โ โ raise _InactiveRpcError(state) # pytype: disable=not-instantiable โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAUTHENTICATED
details = "Received http2 header with status: 401"
debug_error_string = "UNKNOWN:Error received from peer {created_time:"2024-03-24T13:07:36.398921517+01:00", grpc_status:16, grpc_message:"Received http2 header with
status: 401"}"
Cornelis Boon
03/24/2024, 12:40 PM/flyteidl.service.AuthMetadataService
to ingress-nginx's configmap no-auth-locations
However, now getting:
_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNIMPLEMENTED
details = "unknown service flyteidl.service.AuthMetadataService"
debug_error_string = "UNKNOWN:Error received from peer {created_time:"2024-03-24T13:38:14.959242639+01:00", grpc_status:12, grpc_message:"unknown service
flyteidl.service.AuthMetadataService"}"
Cornelis Boon
03/24/2024, 12:49 PMOIDC base URL required when authentication is enabled
which I obviously do not have and it won't allow me to pass an empty string either ๐คCornelis Boon
03/25/2024, 11:21 AMCornelis Boon
03/25/2024, 1:32 PM