https://flyte.org
Join Slack
I am trying flyte deployment on GCP with IAP enabled. I found this open issue <https://github.com/fl...
# flyte-support
h
I am trying flyte deployment on GCP with IAP enabled. I found this open issue https://github.com/flyteorg/flytekit/pull/2034 . It appears that the pyflyte/flytekit is not working with IAP enabled server. Is there a older version of the flytekit I can use in which it was working ?
I see a similar error when I try. The LB logs show 401 errors.
f
cc @cool-lifeguard-49380 was trying to get this working
not sure if folks from spotify or gojek or @agreeable-kitchen-44189 can comment
c
Hey @high-microphone-7046, I forgot to close this draft PR, sorry. I actually got the grpc team to fix the underlying issue in the python client. The newly released grpcio==1.62.0 now behaves the same way e.g. the go client does. I pinned this version in the IAP plugin in this PR. Note: I did add proxy auth support to flyteidl’s admin client so that this integration works also with flytectl but flytectl still uses an older version of flyteidl. A colleague of mine tried to upgrade flyteidl but ran into issues because of flytestdlib. Maybe @freezing-airport-6809 knows who in union knows what needs to be changed because of stdlib?
If you can’t get it to work with the new gprcio version, please let me know, can help you get this to work.
h
I tried the steps : flytekitplugins-identity-aware-proxy 1.10.8b4, grpcio==1.62.0rc1 . Tried the test-iap.py
@cool-lifeguard-49380 do I need to update the auth_interceptor.py?
c
do I need to update the auth_interceptor.py?
No
To me it looks like you don’t sent a valid token. Even with the grpc bug, it shouldn’t shwo 401 🤔
Does the user who creates the token have IAP web app user permission?
f
@cool-lifeguard-49380 let me talk with @high-park-82026 and others - there are too many PRs at the moment
🙏 1
h
@cool-lifeguard-49380 Yes, the user has the web app user permission. I am able to run the proxyCommand (from config.yaml) manually and it logs me in.
c
It would actually do that without you having the permission, this one is only checked once you try to access the protected resource.
401 handled_by_identity_aware_proxy still suggests that there is an issue with permissions. Just to be sure: • Since IAP is activated, can you pls try with Flyte not using any auth itself for now (just to reduce the number of vars) • Do you use the istio setup from the guide you linked? In which time zone are you? We could jump on a quick call
h
Hey @cool-lifeguard-49380 Its working now when I try on a different machine with a GUI. I was previously trying on a Linux VM, there is a subprocess.run in the authenticator.py (Function: CommandAuthenticator) which should bring up the sign in link. That somehow hangs in the background on a Linux VM , I was expecting it to show me a link. On Windows its working fine!
c
Ah yeah that makes sense 😕 I think the CommandAuthenticator should better parse the outputs line by line of the auth sub command and log them unless a line is a token, this would have prevented this
On your linux machine, can you pls try
Copy code
import webbrowser
webbrowser.open_new_tab("<https://www.google.com>")
At least on a local machine i would have expected this to work.
On a GCP compute engine VM, don’t use flyte-iap’s user id token command but the service account one as this one retrieves the token from the metadata server without the need for an interactive auth flow that opens a browser window.
👍 1
7 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by