Hey, I am trying to use the `clientSecret` authflo...
# announcements
h
Hey, I am trying to use the
clientSecret
authflow to be able to build and register workflows in our GitHub Actions CI builds. I looked at the CI docs here, and getting the following error both locally and in our CI when I run the same command. Has anyone experienced anything similar? Flytectl config available in comments
Copy code
☁  ~  flytectl get projects
Error: rpc error: code = Unauthenticated desc = transport: oauth2: cannot fetch token: 401 Unauthorized
Response: {"error":"invalid_client","error_description":"Client authentication failed (e.g., unknown client, no client authentication included, or unsupported authentication method)."}
Copy code
admin:
  endpoint: dns:///domain
  authType: ClientSecret
  clientId: theClientId 
  clientSecretLocation: /path/to/secret # not base64 encoded 
  insecure: false 
logger:
  show-source: true
  level: 1
s
The flow seems right to me. Can you double check if the client ID and secret are correct?
h
They are correct! Using the same ones in the admin. Should I do any encoding?
p
Hi @Hampus Rosvall, Can you check your admin config and if the clientId matches what you are passing. Specifically this section and also from the log seems authServerType might not have been set in your case and defaulting to SelfAuth
Copy code
appAuth:
        authServerType: External
        externalAuthServer:
          baseUrl: https://<provider-url>/oauth2/<auth-server-id>
        thirdPartyConfig:
          flyteClient:
            clientId: <client-id>
            redirectUri: <http://localhost:53593/callback>
            scopes:
            - all
h
Hey, the admin pod is crashlooping after I applied those changes:
Copy code
☁  terraform [main] ⚡ k logs flyteadmin-7fc47f776d-ddn4n
Defaulted container "flyteadmin" out of: flyteadmin, run-migrations (init), seed-projects (init), sync-cluster-resources (init), generate-secrets (init)
time="2022-08-18T07:24:48Z" level=info msg="Using config file: [/etc/flyte/config/cluster_resources.yaml /etc/flyte/config/clusters.yaml /etc/flyte/config/db.yaml /etc/flyte/config/domain.yaml /etc/flyte/config/logger.yaml /etc/flyte/config/remoteData.yaml /etc/flyte/config/server.yaml /etc/flyte/config/storage.yaml /etc/flyte/config/task_resource_defaults.yaml]"
{"json":{},"level":"debug","msg":"Config section [notifications] updated. No update handler registered.","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"debug","msg":"Config section [domains] updated. No update handler registered.","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"debug","msg":"Config section [externalevents] updated. No update handler registered.","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"debug","msg":"Config section [queues] updated. No update handler registered.","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"debug","msg":"Config section [qualityofservice] updated. No update handler registered.","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"debug","msg":"Config section [task_resources] updated. No update handler registered.","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"debug","msg":"Config section [plugins] updated. No update handler registered.","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"debug","msg":"Config section [plugins.catalogcache] updated. No update handler registered.","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"debug","msg":"Config section [plugins.k8s] updated. No update handler registered.","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"debug","msg":"Config section [admin] updated. Firing updated event.","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"debug","msg":"Config section [remotedata] updated. No update handler registered.","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"debug","msg":"Config section [namespace_mapping] updated. No update handler registered.","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"debug","msg":"Config section [database] updated. No update handler registered.","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"debug","msg":"Config section [auth] updated. No update handler registered.","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"debug","msg":"Config section [server] updated. No update handler registered.","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"debug","msg":"Config section [scheduler] updated. No update handler registered.","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"debug","msg":"Config section [cloudevents] updated. No update handler registered.","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"debug","msg":"Config section [secrets] updated. No update handler registered.","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"debug","msg":"Config section [storage] updated. No update handler registered.","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"info","msg":"setting metrics keys to [project domain wf task phase tasktype runtime_type runtime_version app_name]","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"info","msg":"Serving Flyte Admin Insecure","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"info","msg":"Starting profiling server on port [10254]","ts":"2022-08-18T07:24:48Z"}
{"json":{},"level":"error","msg":"Error creating resource server 404 Not Found: ","ts":"2022-08-18T07:24:48Z"}
Error: 404 Not Found:
Usage:
  flyteadmin serve [flags]
Your suggestions seems related to https://docs.flyte.org/en/latest/deployment/cluster_config/auth_setup.html#oauth2-authorization-server, and are not outlined in CI section. Or am I missing something?
p
May be we are missing these things in the CI docs . Can you share your entire auth section of admin config. Do the baseUrl in appAuth match one thats in userAuth
h
Gotcha! No worries, sure one sec
Copy code
server.yaml: |
    auth:
      appAuth:
        authServerType: External
        externalAuthServer:
          baseUrl: <https://login.microsoftonline.com/tenantId/v2.0>
        thirdPartyConfig:
          flyteClient:
            clientId: clientId
            redirectUri: <http://localhost:53593/callback>
            scopes:
            - offline
            - all
      authorizedUris:
      - <https://domain>
      - <http://flyteadmin:80>
      - <http://flyteadmin.flyte.svc.cluster.local:80>
      userAuth:
        openId:
          baseUrl: <https://login.microsoftonline.com/tenantId/v2.0>
          clientId: clientId
          scopes:
          - profile
          - openid
Everything is matching, just masked out the actual Ids!
p
Does this open up on your browser
Copy code
<https://login.microsoftonline.com/tenantId/v2.0/.well-known/oauth-authorization-server>
h
No it doesn’t, only
openid-configuration
suffix works
It returns 302
p
Ok can you add
Copy code
appAuth:
        authServerType: External
        externalAuthServer:
          baseUrl: <https://login.microsoftonline.com/tenantId/v2.0>
          metadataUrl: .well-known/openid-configuration
And check
h
Sure
p
Also if you can search for jwks_uri/jwksUri in .well-know config if it exists
h
jwks_uri is there!
p
cool . then it should not fail there any more with the new config update.
h
Trying now, just restarting pods
The scopes all and offline are not supported my Microsoft
Copy code
☁  terraform [main] ⚡ flytectl get projects
Error: rpc error: code = Unauthenticated desc = transport: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"invalid_scope","error_description":"AADSTS1002012: The provided value for scope offline all is not valid
p
Ok . Seems there is offline_access instead for ms
h
Yeah, in your docs it says that the all scope is required. I could try only the offline_access scope
p
openid, email would also be required
It would be used for checking the logged in user who initiated the request
h
Right, now I get this error
Copy code
☁ ⚡ flytectl get projects
Error: rpc error: code = Unauthenticated desc = transport: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"invalid_scope","error_description":"AADSTS1002012: The provided value for scope openid email offline_access is not valid. Client credential flows must have a scope value with /.default suffixed to the resource identifier
h
Sure, will check it out and let you know how it goes! 🙂 Thanks for your support
Did not work to use that scope either..
p
Strange. what is the scope you added and also is your environment up with flytepropeller being able to communicate with admin .
h
how would I verify that communication?
p
Are you able to lunch any workflow from the flyteconsole UI and see the state changes being updated for the task
i used this scope https://graph.microsoft.com/.default
That seems to be an example scope
h
Yes I can run workflows from UI/flytectl
And see graph of nodes, going from running -> completed etc
Copy code
i used this scope <https://graph.microsoft.com/.default>
That seems to be an example scope
Right, I really just know the oidc/oauth2 flows on a high level so I just tried whatever was there 😄
p
Thats strange. If propeller is able to do it so why can’t flytectl , Can you reuse flytepropeller’s client_id and client_secret and recheck in your original setup where things worked from UI .Change the clientId under flyteClient: to what you have configured for propeller
Apologize but this is our first experience with Azure AD so not sure whats going on.
h
No need to apologise haha, really appreciate the help!
I will check tomorrow :)
👍 1
Copy code
server.yaml: |
    auth:
      appAuth:
        thirdPartyConfig:
          flyteClient:
            clientId: flytectl
            redirectUri: <http://localhost:53593/callback>
            scopes:
            - all
            - offline
      authorizedUris:
      - <https://domain>
      - <http://flyteadmin:80>
      - <http://flyteadmin.flyte.svc.cluster.local:80>
      userAuth:
        openId:
          baseUrl: <https://login.microsoftonline.com/tenantId/v2.0>
          clientId: clientId
          scopes:
          - profile
          - openid
This is my config currently. Why would I change from our IdP clientId to use Flytepropeller? Not quite sure I am following
p
Hi @Hampus Rosvall can we get on a call to resolve this.
h
Hey! Sure we can do that, what’s your availability?
p
I am available now . we can do google meet if you are too
I am available for the next 2 hours .Ping me whenever you are
344 Views