Hey I've setup my cluster with ingress that's basi...
# flyte-supportg
gentle-tomato-480
03/22/2024, 3:03 PMHey I've setup my cluster with ingress that's basic auth secured.
I am not sure how I should setup the config for pyflyte though so I can run/connect to it?
Things I've tried are (and then running `pyflyte run remote-task ..)`:
• set
admin.endpointdns:///user:pass@domainadmin.endpoint<http://user:pass@domain>admin.endpoint<dns://domain>✅ 1
gentle-tomato-480
03/22/2024, 3:41 PMTried setting
admin.endpoint<dns://domain:8089>admin.authorizationHeaderBasic BASE64(user:pass) Copy code
Failed to connect to remote host: FD Shutdowngentle-tomato-480
03/22/2024, 3:45 PMWhen trying to connect via grpcurl, I get:
Copy code
Failed to dial target host "domain:8089": context deadline exceededdomain/consolegentle-tomato-480
03/22/2024, 3:46 PMI do see the domain being resolved to the correct IP, so something is going on. Just don't see anything happening in the ingress or flyte logs
gentle-tomato-480
03/22/2024, 4:10 PMAh. Seems like this is because TLS isn't properly setup 🤔
t
thankful-minister-83577
03/22/2024, 4:12 PM
it’s just dns:///domain
thankful-minister-83577
03/22/2024, 4:12 PM
if you’re using an idp and openidconnect it should just route you to your idp
thankful-minister-83577
03/22/2024, 4:12 PM
if you’re using client credentials you can do something like this https://github.com/flyteorg/flytekit/blob/master/tests/flytekit/unit/configuration/configs/creds_secret_env_var.yaml
g
gentle-tomato-480
03/22/2024, 4:12 PMI'm using simple basic auth for now, hence the auth header
t
thankful-minister-83577
03/22/2024, 4:13 PM
the idea is your secret is inside the
FAKE_SECRET_NAMEg
gentle-tomato-480
03/22/2024, 4:13 PMAh, gotcha
t
thankful-minister-83577
03/22/2024, 4:13 PM
so as to not store a secret inside a config file of course
g
gentle-tomato-480
03/22/2024, 4:15 PMNice that's convenient
gentle-tomato-480
03/22/2024, 4:17 PMI'm planning to store the whole config file in my secret manager and then mount it wherever is necessary. That should be secure enough too, right? OR should I keep the authHeader a separate secret and ideally still use the FAKE_SECRET_NAME env approach?
gentle-tomato-480
03/24/2024, 12:15 PMFollowup, I managed to setup TLS correctly and am now able to hit my flyte deployment with an `endpoint: dns:///my-domain`with no auth configured. However, can't seem to figure out the correct way to configure pyflyte for if I secure my deployment with basic-auth (in nginx ingress).
This is my current config. Still getting 401s when I try to run
pyflyte run remote-task listFlyteRemote Copy code
remote = FlyteRemote(config=Config.auto(), default_project="flytesnacks",default_domain="development")
flyte_wf = remote.fetch_workflow(name="<http://workflows.example.wf|workflows.example.wf>")AuthMetadataService✅ 1
gentle-tomato-480
03/24/2024, 12:17 PM Copy code
╭─────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ───────────────────────────────────────────────────────────────────────╮
│ in <module>:1 │
│ │
│ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/flytekit/remote/remote.py:378 in fetch_workflow │
│ │
│ ❱ 378 │ │ │ self.client.list_workflows_paginated, │
│ │
│ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/flytekit/remote/remote.py:236 in client │
│ │
│ ❱ 236 │ │ │ self._client = SynchronousFlyteClient(self.config.platform, **self._kwargs) │
│ │
│ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/flytekit/clients/raw.py:50 in __init__ │
│ │
│ ❱ 50 │ │ │ cfg, upgrade_channel_to_authenticated(cfg, upgrade_channel_to_proxy_authenti │
│ │
│ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/flytekit/clients/auth_helper.py:140 in upgrade_channel_to_authenticated │
│ │
│ ❱ 140 │ authenticator = get_authenticator(cfg, RemoteClientConfigStore(in_channel)) │
│ │
│ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/flytekit/clients/auth_helper.py:75 in get_authenticator │
│ │
│ ❱ 75 │ │ return ClientCredentialsAuthenticator( │
│ │
│ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/flytekit/clients/auth/authenticator.py:213 in __init__ │
│ │
│ ❱ 213 │ │ cfg = cfg_store.get_client_config() │
│ │
│ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/flytekit/clients/auth_helper.py:38 in get_client_config │
│ │
│ ❱ 38 │ │ public_client_config = metadata_service.GetPublicClientConfig(PublicClientAuthCo │
│ │
│ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/grpc/_interceptor.py:277 in __call__ │
│ │
│ ❱ 277 │ │ response, ignored_call = self._with_call( │
│ │
│ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/grpc/_interceptor.py:332 in _with_call │
│ │
│ ❱ 332 │ │ return call.result(), call │
│ │
│ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/grpc/_channel.py:439 in result │
│ │
│ ❱ 439 │ │ raise self │
│ │
│ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/grpc/_interceptor.py:315 in continuation │
│ │
│ ❱ 315 │ │ │ │ response, call = self._thunk(new_method).with_call( │
│ │
│ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/grpc/_channel.py:1193 in with_call │
│ │
│ ❱ 1193 │ │ return _end_unary_response_blocking(state, call, True, None) │
│ │
│ <root>/anaconda3/envs/flyte/lib/python3.10/site-packages/grpc/_channel.py:1005 in _end_unary_response_blocking │
│ │
│ ❱ 1005 │ │ raise _InactiveRpcError(state) # pytype: disable=not-instantiable │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAUTHENTICATED
details = "Received http2 header with status: 401"
debug_error_string = "UNKNOWN:Error received from peer {created_time:"2024-03-24T13:07:36.398921517+01:00", grpc_status:16, grpc_message:"Received http2 header with
status: 401"}"gentle-tomato-480
03/24/2024, 12:40 PMAh. Solved it by adding
/flyteidl.service.AuthMetadataServiceno-auth-locations Copy code
_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNIMPLEMENTED
details = "unknown service flyteidl.service.AuthMetadataService"
debug_error_string = "UNKNOWN:Error received from peer {created_time:"2024-03-24T13:38:14.959242639+01:00", grpc_status:12, grpc_message:"unknown service
flyteidl.service.AuthMetadataService"}"gentle-tomato-480
03/24/2024, 12:49 PMI'm guessing I need to enable auth for this to work? But if i try that I get:
Copy code
OIDC base URL required when authentication is enabledgentle-tomato-480
03/25/2024, 11:21 AMDecided to start over and just follow https://docs.flyte.org/en/latest/deployment/configuration/auth_setup.html
gentle-tomato-480
03/25/2024, 1:32 PMWhich worked instantly 👍
🚀 2
77 Views