Hi Community I tried to integrate self hosted Gitlab as an e

Question

Hi Community I tried to integrate self hosted Gitlab as an external OAuth2 Authorization Server but stuck at `CSRF TOKEN VALIDATION FAILED` any clues on how to debug the problem The flyte is deployed on self hosted k8s cluster with an internal domain name and HTTP access not HTTPS using template generated by `flyte core` helm chart `helm template flyte core f values yaml` The OIDC seems to work After a fresh deployment when opening flyte console it will redirect to the gitlab authorization page and after approval it can redirect back to flyte console page with username show on the top right corner but when I try to register workflow according to the tutorial 1 with command `flytectl register files project flytesnacks domain development archive flyte package tgz version v1` the gitlab authorization page is prompted successfully but when redirect back to `` page it complains about `401` error And the log of `flyteadmin` shows something like ``` json level error msg Invalid CSRF token cookie CSRF TOKEN VALIDATION FAILED CSRF token does not match state 2r4rcd3npg 3237e1083ec0ae2bd20acbe8a5817d18475faaee5a060d2184ab7ffddd151290 vs OXpoczQyanRxcW43c3hnZ3RjbnBnZjZrNnptMnA2dDY ts 2023 01 29T06 43 34Z ``` There are several small questions that might be relevant 1 In the auth doc 2 there is a comment for `redirectUri ` saying that `This should not change` but I suppose it is for sandbox deployment so I changed it to the domain name something like `` Otherwise it will complains about invalid redirect uri when authorize on gitlab page 2 For the `scopes` according to the gitlab doc 3 I only set it to `read user` 3 Any approach to show more logs for `flytectl` CLI I tried to set the log level to `0` but it does not show anything 4 Any approach to show more logs for `flyteadmin` service I found there is a `flyteadmin extraArgs` in the template but does not know how to inject ` logger level 0` The relevant `values yaml` for auth is shown as following with some sensitive info masked ```configmap adminServer auth appAuth authServerType External externalAuthServer metadataUrl well known openid configuration thirdPartyConfig flyteClient clientId lt client id generated by gitlab gt redirectUri scopes read user authorizedUris userAuth openId baseUrl clientId lt client id generated by gitlab gt scopes openid server security useAuth true flyteadmin secrets oidc client secret lt client secret generated by gitlab gt secrets adminOauthClientCredentials clientId lt client id generated by gitlab gt clientSecret lt client secret generated by gitlab gt ``` 1 2 3

Ketan (kumare3) · Answer

The number of auth integrations is crazy But I know someone else is using gitlab

Xuan Hu · Answer

Will it be possible have some configuration examples or clues to debug the problem

Ketan (kumare3) · Answer

Let me add a few auth experts here

Ketan (kumare3) · Answer

Cc < Prafulla Mahindrakar> < Haytham Abuelfutuh>

Xuan Hu · Answer

Another information is that we are using HAProxy instead of Nginx for Ingress that might be the reason for the csrf token error since I am not an expert I have already involved one of my colleagues for help Just want to confirm the config looks good and if there is any recommendation to debug the problem

Xuan Hu · Answer

Sorry I missed one import aspect the client configuration The `config yaml` I use now is ```admin authType Pkce endpoint dns flyte example com insecure false insecureSkipVerify true logger level 0 show source true``` With this config the gitlab authorization page can be prompted successfully but `401` error after callback to `flyteadmin` I also tried the `ClientSecret` type but failed It complains about `JWT VERIFICATION FAILED` or `500` error for gitlab The gitlab authorization page will not shown

Prafulla Mahindrakar · Answer

Hi < Xuan Hu> please check my comments on these gt In the auth doc 2 there is a comment for `redirectUri ` saying that `This should not change` but I suppose it is for sandbox deployment so I changed it to the domain name something like `` This needs to remain the same `` as this is the url where flytectl recives a callback to exchange auth code for the token in PKCE flow started by it So this would remain same irrespective of sandbox gt Otherwise it will complains about invalid redirect uri when authorize on gitlab page You don t have to configure this callback url in gitlab Can you more details here gt For the `scopes` according to the gitlab doc 3 I only set it to `read user` Currently its required according to our current set of supported providers to define scope name `all` other providers like okta support adding new scopes for oauth apps gt Any approach to show more logs for `flytectl` CLI I tried to set the log level to `0` but it does not show anything You can pass arguments logger level 6 gt Any approach to show more logs for `flyteadmin` service I found there is a `flyteadmin extraArgs` in the template but does not know how to inject ` logger level 0` you can edit you flyte admin base configmap and add logger configuration with level 6 ``` logger yaml | logger level 6```

Xuan Hu · Answer

Hi < Prafulla Mahindrakar> thanks for the detailed reply I will try it soon and come back then

Xuan Hu · Answer

Hi < Prafulla Mahindrakar> I am still a little confused about the `redirectUri` part Let us assume the flyte is deployed with an internal domain name for example `` The `redirectUri` need to be configured for both `flyteadmin` and gitlab Application 1 And during the OAuth process 2 web browser will firstly redirect to the Gitlab authorization page And after clicking `Authorize` it will redirect back to the configured `redirectUri` If we use `` then it will be an invalid address since the service is deploy with `` instead Any comments 1 2

Samhita Alla · Answer

Are you seeing any error when the `redirectUri` is `<http localhost 53593 callback>`

Xuan Hu · Answer

Yes firstly in the configuration of Gitlab Application I have to set the Redirect URI to the same one a k a `` otherwise it will complains about invalid redirect uri And then after I click the `Authorize` button when being prompted to Gitlab Authorization page it will redirect back to `` in web browser which is a invalid address since the flyte is deployed on the server

Prafulla Mahindrakar · Answer

Hi < Xuan Hu> Sorry for the confusion Seems the docs would need to be improved here There are three aoauth gitlab apps you need to configure to use with flyte Repasting the configmap you posted along with places where these clientId are defined and there purpose ```configmap adminServer auth appAuth authServerType External externalAuthServer metadataUrl well known openid configuration thirdPartyConfig flyteClient clientId lt client id generated by gitlab gt = gt This clientId is used for flytectl client and should be configured as native ouath app redirectUri = gt the redirectURI here should be since flytectl starts a http server to receive the callback in this flow scopes read user authorizedUris userAuth openId baseUrl clientId lt client id generated by gitlab gt = gt this clientId is used for flyteconsole or UI client which uses client credentials flow and it secret is stored in oidc client secret The redirect URI for this app should be or wherever your lt flyte domain is configured gt callback scopes openid server security useAuth true flyteadmin secrets oidc client secret lt client secret generated by gitlab gt = gt used as secret for openId clientId secrets adminOauthClientCredentials clientId lt client id generated by gitlab gt = gt this clientId is used by flytepropeller and scheduler this should also use the same callback used by flyteconsole or wherever your lt flyte domain is configured gt callback clientSecret lt client secret generated by gitlab gt = gt secret for the above clientId```

Prafulla Mahindrakar · Answer

the redirectURI <http localhost 53593 callback> is only to be configured for flytectl app and when you run any command from flytectl it will open a browser to do authorization using gitlab while the browser is running flytectl will be running a http callback server on port 53593 and when you complete the authorization flow in the browser it will redirect to this http callback endpoint and exchange the auth token with the gitlab

Prafulla Mahindrakar · Answer

We can jump on a call if its easier and we can coordinate on a time

Xuan Hu · Answer

Thanks < Prafulla Mahindrakar> for the detailed explanation will come back after some further investigation

Xuan Hu · Answer

Hi < Prafulla Mahindrakar> good news is that I can see authentication result page when using flytectl with `Pkce` auth type but the auth process still does not work The config is something like ```admin authType Pkce endpoint dns flyte example com insecure false insecureSkipVerify true``` And the error log is ``` json src token source provider go 140 level error msg Error fetching token using auth flow due to error while exchanging auth code due to oauth2 cannot fetch token 401 Unauthorized nResponse error invalid client error description Client authentication failed due to unknown client no client authentication included or unsupported authentication method ts 2023 01 31T18 07 09+08 00 Error authentication error Original Error rpc error code = Unauthenticated desc = token parse error JWT VERIFICATION FAILED Could not retrieve id token from metadata caused by rpc error code = Unauthenticated desc = Request unauthenticated with IDToken Auth Error error while exchanging auth code due to oauth2 cannot fetch token 401 Unauthorized Response error invalid client error description Client authentication failed due to unknown client no client authentication included or unsupported authentication method json src main go 13 level error msg authentication error Original Error rpc error code = Unauthenticated desc = token parse error JWT VERIFICATION FAILED Could not retrieve id token from metadata caused by rpc error code = Unauthenticated desc = Request unauthenticated with IDToken Auth Error error while exchanging auth code due to oauth2 cannot fetch token 401 Unauthorized nResponse error invalid client error description Client authentication failed due to unknown client no client authentication included or unsupported authentication method ts 2023 01 31T18 07 09+08 00 ``` BTW I failed to find any place to set the `client secret` for `flytectl` in helm chart template that may be the cause for the unauthenticated problem I also tried `ClientSecret` auth type but it will get 500 internal error from gitlab The config is something like ```admin authType ClientSecret clientId lt client id for flytectl gt clientSecretLocation lt path of client secret for flytectl gt endpoint dns flyte example com insecure false insecureSkipVerify true scopes openid read user ``` The relevant error log is ``` json src main go 13 level error msg Connection Info Endpoint dns flyte example com InsecureConnection false AuthMode ClientSecret rpc error code = Unauthenticated desc = transport per RPC creds failed due to error oauth2 cannot fetch token 500 Internal Server Error nResponse u003c DOCTYPE html u003e n u003chtml u003e n u003chead u003e n u003cmeta content= width=device width initial scale=1 maximum scale=1 name= viewport u003e n u003ctitle u003eSomething went wrong 500 u003c title u003e n u003cstyle u003e n body n color 666 n text align center n font family Helvetica Neue Helvetica Arial sans serif n margin auto n font size 14px n n n h1 n font size 56px n line height 100px n font weight 400 n color 456 n n n h2 n font size 24px n color 666 n line height 1 5em n n n h3 n color 456 n font size 20px n font weight 400 n line height 28px n n n hr n max width 800px n margin 18px auto n border 0 n border top 1px solid EEE n border bottom 1px solid white n n n img n max width 40vw n display block n margin 40px auto n n n a n line height 100px n font weight 400 n color 4A8BEE n font size 18px n text decoration none n n n container n margin auto 20px n n n go back n display none n n n u003c style u003e n u003c head u003e n n u003cbody u003e n u003ca href= u003e n u003cimg src= data image svg+xml base64 PHN2ZyB3aWR0aD0iMjEwIiBoZWlnaHQ9IjIxMCIgdmlld0JveD0iMCAwIDIxMCAyMTAiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+CiAgPHBhdGggZD0iTTEwNS4wNjE0IDIwMy42NTVsMzguNjQtMTE4LjkyMWgtNzcuMjhsMzguNjQgMTE4LjkyMXoiIGZpbGw9IiNlMjQzMjkiLz4KICA8cGF0aCBkPSJNMTA1LjA2MTQgMjAzLjY1NDhsLTM4LjY0LTExOC45MjFoLTU0LjE1M2w5Mi43OTMgMTE4LjkyMXoiIGZpbGw9IiNmYzZkMjYiLz4KICA8cGF0aCBkPSJNMTIuMjY4NSA4NC43MzQxbC0xMS43NDIgMzYuMTM5Yy0xLjA3MSAzLjI5Ni4xMDIgNi45MDcgMi45MDYgOC45NDRsMTAxLjYyOSA3My44MzgtOTIuNzkzLTExOC45MjF6IiBmaWxsPSIjZmNhMzI2Ii8+CiAgPHBhdGggZD0iTTEyLjI2ODUgODQuNzM0Mmg1NC4xNTNsLTIzLjI3My03MS42MjVjLTEuMTk3LTMuNjg2LTYuNDExLTMuNjg1LTcuNjA4IDBsLTIzLjI3MiA3MS42MjV6IiBmaWxsPSIjZTI0MzI5Ii8+CiAgPHBhdGggZD0iTTEwNS4wNjE0IDIwMy42NTQ4bDM4LjY0LTExOC45MjFoNTQuMTUzbC05Mi43OTMgMTE4LjkyMXoiIGZpbGw9IiNmYzZkMjYiLz4KICA8cGF0aCBkPSJNMTk3Ljg1NDQgODQuNzM0MWwxMS43NDIgMzYuMTM5YzEuMDcxIDMuMjk2LS4xMDIgNi45MDctMi45MDYgOC45NDRsLTEwMS42MjkgNzMuODM4IDkyLjc5My0xMTguOTIxeiIgZmlsbD0iI2ZjYTMyNiIvPgogIDxwYXRoIGQ9Ik0xOTcuODU0NCA4NC43MzQyaC01NC4xNTNsMjMuMjczLTcxLjYyNWMxLjE5Ny0zLjY4NiA2LjQxMS0zLjY4NSA3LjYwOCAwbDIzLjI3MiA3MS42MjV6IiBmaWxsPSIjZTI0MzI5Ii8+Cjwvc3ZnPgo= n alt= GitLab Logo u003e n u003c a u003e n u003ch1 u003e n 500 n u003c h1 u003e n u003cdiv class= container u003e n u003ch3 u003eWhoops something went wrong on our end u003c h3 u003e n u003chr u003e n u003cp u003eTry refreshing the page or going back and attempting the action again u003c p u003e n u003cp u003ePlease contact your GitLab administrator if this problem persists u003c p u003e n u003ca href= javascript history back class= js go back go back u003eGo back u003c a u003e n u003c div u003e n u003cscript u003e n function n var goBack = document querySelector js go back n n if history length u003e 1 n goBack style display = inline n n n u003c script u003e n u003c body u003e n u003c html u003e n ts 2023 01 31T16 05 02+08 00 ``` Any comments are welcome And Thanks in advance

Prafulla Mahindrakar · Answer

Hi < Xuan Hu> Seems the gitlab oauth app is misconfigured according to the errors you pasted PKCE based apps don t have clientSecret Seems the app has been incorrectly been created in gitlab I couldn t find the documentation from gitlab docs to create such apps but this is corresponding reference from OKTA The error `Client authentication failed due to unknown client no client authentication included or unsupported authentication method` indicates this misconfiguration Seems it attempting to do a PKCE flow on an gitlab oauth app which has been created as client secret app When you are using ClientSecret of flytectl and getting 500 error which is again not an expected flow and hence it might be throwing this from gitlab I would suggest getting right docs to setup gitlab oauth PKCE app no client secret Other option is to use client secret based app in which you can try reusing the clientId and secret which is used for flytepropeller and scheduler i e this one ``` adminOauthClientCredentials clientId lt client id generated by gitlab gt clientSecret lt client secret generated by gitlab gt ``` The second option is suited for CI usescases where you just have CI server authenticating with gitlab whereas if you are deploying flytectl to multiple users machines with browser access PKCE based oauth is the preferred approach

Xuan Hu · Answer

Hi < Prafulla Mahindrakar> thanks for the explanation I have to admit I am not an expert for OAuth I will try to investigate more Thanks again for all the patient help

Prafulla Mahindrakar · Answer

Np Unfortunately gitlab as auth provider hasn t been tried in the flyte community yet but but let us know how it goes and will be there to help you on it

Xuan Hu · Answer

Hi < Prafulla Mahindrakar> thanks for the kindness Just to update the latest progress For `Pkce` auth type Good news is that I can see the successful authentication result page from `localhost` with info `Cool Your authentication was successful and you can close the window ` The key point for no client secret authentication is to uncheck the `Confidential` option refer to the attached screenshot But the auth process still has some problem The error log is something like below ``` json src auth flow orchestrator go 77 level info msg Opening the browser at lt client id for flytectl with localhost redirect uri gt u0026redirect uri=http 3A 2F 2Flocalhost 3A53593 2Fcallback u0026response type=code u0026scope=email+openid+profile+read user u0026code challenge=gYYRv7I Eo2E3xiEbF2iSaIJl5hVVOeXJBzI0NQ 6l4 u0026code challenge method=S256 u0026nonce=cG1sd2NobG5zNzYycjJrY3ZqeDRrN3ZsNWQ1NDdkang u0026state=cTduc242bTZja3FuNWdtczJ2ZmNuMmI2czdoc3hkdHo ts 2023 02 02T11 17 00+08 00 Error Connection Info Endpoint dns flyte example com InsecureConnection false AuthMode Pkce rpc error code = Unauthenticated desc = token parse error JWT VERIFICATION FAILED Could not retrieve id token from metadata caused by rpc error code = Unauthenticated desc = Request unauthenticated with IDToken json src main go 13 level error msg Connection Info Endpoint dns flyte example com InsecureConnection false AuthMode Pkce rpc error code = Unauthenticated desc = token parse error JWT VERIFICATION FAILED Could not retrieve id token from metadata caused by rpc error code = Unauthenticated desc = Request unauthenticated with IDToken ts 2023 02 02T11 17 10+08 00 ``` I searched on the Google and find a StackOverflow answer 1 saying that `email` scope need to be checked but I tried it with even more scopes and it does not work For the `ClientSecret` auth type I tried to reuse the `client id` and `client secret` for flytepropeller and scheduler but it still get 500 error from gitlab For reminder the auth related `values yaml` for `flyte core` helm chart template I use now is ``` configmap adminServer auth appAuth authServerType External externalAuthServer metadataUrl well known openid configuration thirdPartyConfig flyteClient clientId lt client id for flytectl and SAME with flyteproerller and scheduler for ClientSecret auth type gt redirectUri scopes email openid profile read user authorizedUris userAuth openId baseUrl clientId lt client id for flyteconsole gt scopes email openid profile read user server security useAuth true flyteadmin secrets oidc client secret lt client secret for flyteconsole gt secrets adminOauthClientCredentials clientId lt client id for flytepropeller and scheduler gt clientSecret lt client secret for flytepropeller and scheudler gt ``` the client config for `Pkce` and `ClientSecret` is almost the same as before ```admin authType Pkce endpoint dns flyte example com insecure false insecureSkipVerify true``` and ```admin authType ClientSecret clientId lt SAME client id for flytepropeller and scheduler gt clientSecretLocation etc secrets client secret endpoint dns flyte example com insecure false insecureSkipVerify true``` 1

Xuan Hu · Answer

Also for the gitlab applications I created three right now 1 flytectl only used for Pkce client authentication a Redirect URI b Confidiential Unchecked c Scopes read user openid profile email 2 flyteconsole a Redirect URI b Confidential Checked c Scopes read user openid profile email 3 flytepropeller scheduler a Redirect URI tried both localhost and here b Condifential Checked c Scopes read user openid profile email

Xuan Hu · Answer

BTW `ClientSecret` auth type should be a must have option since we have scenarios for CICD e g Daily Test auto finetune with incremental data Also algorithm engineers tend to work on server over ssh with no easy though it could be done via ssh forward web browser access

Ketan (kumare3) · Answer

< Xuan Hu> in short are you still stuck

Xuan Hu · Answer

Hi < Ketan kumare3 > thanks for the caring < Prafulla Mahindrakar> is helping me on that Currently seems there needs a new release of flytectl with the latest flyteidl library We will continue on this tomorrow It is not a hurry blocking issue so we will just keep tracking on it

Prafulla Mahindrakar · Answer

From flyteadmin logs which was shared we saw that authserver is sending opaque token and hence parsing of the token is failing since its not in JWT format and this causes flyteadmin to reject the token ``` json src handlers go 248 level info msg Failed to parse Access Token from context Will attempt to find IDToken Error oidc malformed jwt square go jose compact JWS format must have three parts ts 2023 02 02T06 34 38Z json src token go 103 level deb``` This has happened in the past when the authserver is not sent the right audience config and which has been fixed to allow setting this from client config in flytectl We dont face this internally since we have okta auth server with default audience set to hosted flyte endpoint Certain auth providers like gitlab and auth0 dont have an option to set this default audience and expect the client to send in this when requesting the auth token We will upgrade flytectl with latest flyteidl and test this out with passing audience config and take it from there

Ketan (kumare3) · Answer

I saw you merged do we think that s it

Prafulla Mahindrakar · Answer

< Xuan Hu> We can do `flytectl upgrade` and use the config with audience Also this change to the audience would need to be done to all flyteadmin clients ie scheduler propeller and later flytekit too PR in progress for flytekit We can first verify that this audience issue and then update the values file for propeller and admin to use the same

Ketan (kumare3) · Answer

Cc < Xuan Hu>

Xuan Hu · Answer

Thanks < Ketan kumare3 > and < Prafulla Mahindrakar> I will try it later Suppose to reply early next week

Xuan Hu · Answer

Hi < Prafulla Mahindrakar> I tried the new version `0 6 27` of flytectl and it seems to be the same error Config for flytectl ```admin authType Pkce endpoint dns flyte example com insecure false insecureSkipVerify true audience logger level 6 show source true``` Error log for flytectl ``` json src auth flow orchestrator go 77 level info msg Opening the browser at u0026redirect uri=http 3A 2F 2Flocalhost 3A53593 2Fcallback u0026response type=code u0026scope=email+openid+profile+read user u0026code challenge=c6ja 6ytu4dOTvNfvwgwjnjOmw10niejG7To47Jqymw u0026code challenge method=S256 u0026nonce=bDkyeGo2c25jdmZucDYyNDhnbDJzbGhuanI1aGd6MmI u0026state=aHo1ZHh4ZHp2NGZxaGhkdGJycHZzZ240bHhoZDh2dHQ ts 2023 02 06T13 17 41+08 00 json src version go 103 level debug msg Failed to get version of control plane rpc error code = Unauthenticated desc = token parse error JWT VERIFICATION FAILED Could not retrieve id token from metadata caused by rpc error code = Unauthenticated desc = Request unauthenticated with IDToken n ts 2023 02 06T13 17 46+08 00 json src version go 81 level debug msg rpc error code = Unauthenticated desc = token parse error JWT VERIFICATION FAILED Could not retrieve id token from metadata caused by rpc error code = Unauthenticated desc = Request unauthenticated with IDToken ts 2023 02 06T13 17 46+08 00 ``` ConfigMap for flyteadmin the `values yaml` ```configmap adminServer auth appAuth authServerType External externalAuthServer allowedAudience metadataUrl well known openid configuration thirdPartyConfig flyteClient audience clientId 88151b597efcd5adbfe86fcf54c2a3eb685daa552a41f4bfe4888b95bb10c571 redirectUri scopes email openid profile read user authorizedUris userAuth openId baseUrl clientId 3483bdaa483ebe93638fe637481013c5ca869dd67dceaf45ccf6351ed61ba0ca scopes email openid profile read user server security useAuth true ``` Error log for flyteadmin ``` json src handlers go 238 level debug msg Running authentication gRPC interceptor ts 2023 02 06T05 17 41Z json src token go 83 level debug msg Could not retrieve bearer token from metadata rpc error code = Unauthenticated desc = Request unauthenticated with Bearer ts 2023 02 06T05 17 41Z json src handlers go 248 level info msg Failed to parse Access Token from context Will attempt to find IDToken Error JWT VERIFICATION FAILED Could not retrieve bearer token from metadata caused by rpc error code = Unauthenticated desc = Request unauthenticated with Bearer ts 2023 02 06T05 17 41Z json src token go 103 level debug msg Could not retrieve id token from metadata rpc error code = Unauthenticated desc = Request unauthenticated with IDToken ts 2023 02 06T05 17 41Z json src handlers go 194 level debug msg gRPC server info in logging interceptor method flyteidl service AuthMetadataService GetPublicClientConfig n ts 2023 02 06T05 17 41Z json src handlers go 194 level debug msg gRPC server info in logging interceptor method flyteidl service AuthMetadataService GetOAuth2Metadata n ts 2023 02 06T05 17 41Z json src handlers go 194 level debug msg gRPC server info in logging interceptor method flyteidl service AuthMetadataService GetPublicClientConfig n ts 2023 02 06T05 17 41Z json src handlers go 210 level debug msg Found existing metadata Bearer e54a72ad7f9e782bdcd0febfd146dc83f5b2340a789892ada5808c58df58eadb ts 2023 02 06T05 17 46Z json src handlers go 238 level debug msg Running authentication gRPC interceptor ts 2023 02 06T05 17 46Z json src handlers go 248 level info msg Failed to parse Access Token from context Will attempt to find IDToken Error oidc malformed jwt square go jose compact JWS format must have three parts ts 2023 02 06T05 17 46Z json src token go 103 level debug msg Could not retrieve id token from metadata rpc error code = Unauthenticated desc = Request unauthenticated with IDToken ts 2023 02 06T05 17 46Z ```

Xuan Hu · Answer

Besides I can confirm the token is valid The following command can get valid response ```$ curl header Authorization Bearer e54a72ad7f9e782bdcd0febfd146dc83f5b2340a789892ada5808c58df58eadb <https git example com oauth token info> ```

Prafulla Mahindrakar · Answer

Hi < Xuan Hu> seems from the above curl command you shared that the token return by your configured gitlab identity provider is not issuing JWT tokens Identity tokens are used by flyteadmin to know the users identity info once he is authenticated and verify the Identity tokens signature Since in your case the the auth provider is not issuing JWT and hence it failing in flyteadmin to parse this token This is the function which calls to verify the format for token Can you check your gitlab setup to see what it takes for it to issue JWT tokens Also according to RFC these tokens need to be JWT tokens

Xuan Hu · Answer

Hi < Prafulla Mahindrakar> thanks for the reply and this is the OAuth2 API for Gitlab 1 seems there is no option for jwt token can we conclude that at least currently gitlab is not supported Again I am not an expert of OAuth just want to have a double confirm 1

Xuan Hu · Answer

According to the Overview of Authentication in Flyte 1 oidc is used for user s authentication while OAuth2 is used for client so I suppose we might support access token besides jwt Also according to the log there is an error log `Could not retrieve bearer token from metadata` will this be the actual cause Which should be related to this code snippet 2 but since I am not familiar with Go I failed to find anything unusual 1 2