Hi all, We have our flyte deployment configured to work with Google oauth. All worked well so far, but we're seeing that some new users can't log in: they get a 502 after the

/callback

redirect back from Google. Most users (who previously interacted with the deployment) still have access, without problems - they can log out and back in, also in incognito / different browser. It seems to be connected to the specific Google user accounts. These accounts have no issues with any other app using oidc though.

flyteadmin

prints this in the logs for the failed attempts:

{
  "json": {},
  "level": "error",
  "msg": "Failed to retrieve tokens from request, redirecting to login handler. Error: [EMPTY_OAUTH_TOKEN] Failure to retrieve cookie [flyte_idt], caused by: http: named cookie not present",
  "ts": "2023-07-13T15:25:13Z"
}

I tried so far: • clearing cookies for the "broken" users, trying different browser • restarting all flyte components Any idea what can be causing this issue? What else can I clear / restart to debug further?

Could you take a look at this thread? https://discuss.flyte.org/t/2197578/opened-pr-2746#97daa7fe-3d17-4600-b341-64314ea89e89

@Tomasz Sodzawiczny this is a bit strange, especially if it's only for some users. There's an update to the auth docs (currently in review) that you could check to verify that it matches your deployment: https://flyte--3837.org.readthedocs.build/en/3837/deployment/configuration/auth_setup.html#apply-external-auth-server-configuration

Thanks Samhita I checked out the discussion, it looks like the settings mentioned there are correct in our setup

I've been using this setting in

$HOME/.flyte/config.yaml

logger:
  show-source: true
  level: 6

but not sure about the effect on flyteadmin particularly

Hey, Bumping this one as we are having the exact same problem, we also started to have some of ours users getting

502

even though they could connect to it and use it before. Do you have any ideas on that one?

@Tomasz Sodzawiczny did you figure this out, it seems strange

Ok I think we found the solution. We had this error in our nginx component

2023/08/10 13:36:54 [error] 49#49: *861071 upstream sent too big header while reading response header from upstream,  host: "<http://console.flyte.xxx|console.flyte.xxx>", referrer: "<https://console.flyte.xxx/>"

We added

proxy-buffer-size: "128k"

to the configmap of that deployment and it seems to have solved the problem. cc @Franco Bocci for figuring this out

This is great @Stephen and @Franco Bocci. Thanks for sharing! @Tomasz Sodzawiczny please let us know if if helps with your issue