https://flyte.org logo
#ask-the-community
Title
# ask-the-community
u

盛毅

06/26/2023, 1:53 AM
i installed the flyteorg/flyte-binary v1.7.0 helm chart in my local k8s cluster, when enabled the auth, the flyte container can not started , the error is "{"json":{"src":"start.go:58"},"level":"fatal","msg":"Flyte native scheduler failed to start due to async future was canceled","ts":"2023-06-26T014846Z"}" how to deal with it??
s

Samhita Alla

06/26/2023, 5:15 AM
d

David Espejo (he/him)

06/26/2023, 11:17 AM
Hi @盛毅 Could you share your values file here? Also for auth, what IdP you plan on using? Flyte ships with an internal auth server, do you plan on using it or relay on the custom auth server from your IdP?
u

盛毅

06/27/2023, 1:10 AM
auth:
# enabled Enable Flyte authentication enabled: true # enableAuthServer Enable built-in authentication server enableAuthServer: true # oidc OIDC configuration for Flyte authentication oidc: # baseUrl URL for OIDC provider baseUrl: "http://10.24.50.130:31157/auth/realms/flyte" # clientId Flyte application client ID clientId: "flyte" # clientSecret Flyte application client secret clientSecret: "3v0q10VDcMU9gCYWAaEnZZ7AHiEAz2RO" # internal Configuration for internal authentication # The settings for internal still need to be defined if you wish to use an external auth server # These credentials are used during communication beteween the FlyteAdmin and Propeller microservices internal: # clientId Client ID for internal authentication - set to flytepropeller or external auth server clientId: flytepropeller # clientSecret Client secret for internal authentication clientSecret: "3v0q10VDcMU9gCYWAaEnZZ7AHiEAz2RO" # clientSecretHash Bcrypt hash of clientSecret clientSecretHash: "M3YwcTEwVkRjTVU5Z0NZV0FhRW5aWjdBSGlFQXoyUk8K" # Uncomment next line if needed - set this field if your external Auth server (ex. Auth0) requires an audience parameter # audience: "" # flyteClient Configuration for Flyte client authentication flyteClient: # clientId Client ID for Flyte client authentication clientId: flytectl # redirectUri Redirect URI for Flyte client authentication redirectUri: http://localhost:53593/callback # scopes Scopes for Flyte client authentication scopes: - all # audience Audience for Flyte client authentication audience: "" # authorizedUris Set of URIs that clients are allowed to visit the service on authorizedUris: []
this is the auth part of the valus file, i used the latest version keycloak, this is the error from keycloak log: type=CODE_TO_TOKEN_ERROR, realmId=84370e80-0dad-49a4-87cb-db9e7fd14e3b, clientId=flyte, userId=null, ipAddress=10.244.159.0, error=invalid_code, grant_type=authorization_code, code_id=6ad28d56-0eb4-433c-b715-f551639180c6, client_auth_method=client-secret
d

David Espejo (he/him)

06/27/2023, 6:51 PM
a couple of suggestions for your auth values: 1. Check your
clientSecretHash
. It's usually generated by using:
pip install bcrypt && python -c 'import bcrypt; import base64; print(base64.b64encode(bcrypt.hashpw("<your-client-secret>".encode("utf-8"), bcrypt.gensalt(6))))'
When I tried with your clientSecret, it produced a different output than what you have in the values file (not sure if this is only dummy data) 2. Add the `authorizedUris`value to be an exact match of your Flyte ingress-backed URL, which should also be the same as your configured login redirect URI configured at Keycloak, without
/callback
3. In the `flyteClient`section under `scopes`add
- offline
u

盛毅

06/28/2023, 1:54 AM
hi @David Espejo (he/him), just updated the helm yaml file ,still no luck , here is the auth configuration file generated by helm :
Copy code
auth:
  appAuth:
    selfAuthServer:
      staticClients:
        flytepropeller:
          client_secret: "JDJiJDA2JFdrSEo1UzZ5Rkcud1ZjakNqZHcyTC5Uc1Q3RGs2Q3hhMGNhOGd1YldtdmlQRnZ2ZlRmTE1D"
          grant_types:
          - refresh_token
          - client_credentials
          id: flytepropeller
          response_types:
          - token
          scopes:
          - all
          - offline
          - access_token
        flyte-cli:
          grant_types:
          - refresh_token
          - authorization_code
          id: flyte-cli
          public: true
          redirect_uris:
          - <http://localhost:53593/callback>
          - <http://localhost:12345/callback>
          response_types:
          - code
          - token
          scopes:
          - all
          - offline
          - access_token
        flytectl:
          grant_types:
          - refresh_token
          - authorization_code
          id: flytectl
          public: true
          redirect_uris:
          - <http://localhost:53593/callback>
          - <http://localhost:12345/callback>
          response_types:
          - code
          - token
          scopes:
          - all
          - offline
          - access_token
    thirdPartyConfig:
      flyteClient: 
        audience: ""
        clientId: flytectl
        redirectUri: <http://localhost:53593/callback>
        scopes:
        - all
        - offline
  authorizedUris:
  - <http://flyte.nginx.k8s:30699>
  - <http://flyte-flyte-binary:8088>
  - <http://flyte-flyte-binary.flyte:8088>
  - <http://flyte-flyte-binary.flyte.svc:8088>
  - <http://flyte-flyte-binary.flyte.svc.cluster.local:8088>
  userAuth:
    openId:
      baseUrl: "<http://10.24.50.130:31157/realms/flyte>"
      clientId: "flyte"
      scopes:
      - profile
      - openid
server:
  security:
    useAuth: true
@David Espejo (he/him) this is the keycloak log, i have no idea if it is the keycloak that caused the forbidden error`2023-06-27T220307.166817374-04:00 2023-06-28 020307,166 WARN [org.keycloak.events] (executor-thread-262) type=CODE_TO_TOKEN_ERROR, realmId=84370e80-0dad-49a4-87cb-db9e7fd14e3b, clientId=flyte, userId=null, ipAddress=10.244.159.0, error=invalid_code, grant_type=authorization_code, code_id=a8212ab4-8707-4fec-8302-a34b9d3bd01e, client_auth_method=client-secret` 2023-06-27T220307.320876911-04:00 2023-06-28 020307,320 WARN [org.keycloak.protocol.oidc.utils.OAuth2CodeParser] (executor-thread-262) Code 'b5534144-da1b-46a9-a506-f509de5f0e21' already used for userSession 'a8212ab4-8707-4fec-8302-a34b9d3bd01e' and client '4fe685d9-04ad-4a2d-9421-1033872d5ddc'. 2023-06-27T220307.321208967-04:00 2023-06-28 020307,321 WARN [org.keycloak.events] (executor-thread-262) type=CODE_TO_TOKEN_ERROR, realmId=84370e80-0dad-49a4-87cb-db9e7fd14e3b, clientId=flyte, userId=null, ipAddress=10.244.159.0, error=invalid_code, grant_type=authorization_code, code_id=a8212ab4-8707-4fec-8302-a34b9d3bd01e, client_auth_method=client-secret 2023-06-27T220307.474006816-04:00 2023-06-28 020307,473 WARN [org.keycloak.protocol.oidc.utils.OAuth2CodeParser] (executor-thread-262) Code 'b5534144-da1b-46a9-a506-f509de5f0e21' already used for userSession 'a8212ab4-8707-4fec-8302-a34b9d3bd01e' and client '4fe685d9-04ad-4a2d-9421-1033872d5ddc'. 2023-06-27T220307.474249811-04:00 2023-06-28 020307,474 WARN [org.keycloak.events] (executor-thread-262) type=CODE_TO_TOKEN_ERROR, realmId=84370e80-0dad-49a4-87cb-db9e7fd14e3b, clientId=flyte, userId=null, ipAddress=10.244.159.0, error=invalid_code, grant_type=authorization_code, code_id=a8212ab4-8707-4fec-8302-a34b9d3bd01e, client_auth_method=client-secret
d

David Espejo (he/him)

06/28/2023, 11:48 AM
the main difference I find compared with my deployment (EKS with Okta working), is in the
authorizedUris
field I'm using the domain name without ports
Copy code
authorizedUris:
  - <https://flyte-the-hard-way.uniondemo.run>
  - <http://flyte-backend-flyte-binary:8088>
  - <http://flyte-backend-flyte-binary.flyte:8088>
  - <http://flyte-backend-flyte-binary.flyte.svc:8088>
  - <http://flyte-backend-flyte-binary.flyte.svc.cluster.local:8088>
accordin to the command output you shared in the other thread, your nginx ingress controller is exposing port 80 According to their docs, there should be at least these two annotations to make sure nginx routes gRPC properly:
<http://nginx.ingress.kubernetes.io/ssl-redirect|nginx.ingress.kubernetes.io/ssl-redirect>: "true"
<http://nginx.ingress.kubernetes.io/backend-protocol|nginx.ingress.kubernetes.io/backend-protocol>: "GRPC"
u

盛毅

06/29/2023, 1:11 AM
hi @David Espejo (he/him) , i got it running , the nginx pod loged an error: upstream sent too big header while reading response header. i updated the ingress-nginx-controller configmap ,set proxy-buffer-size: 16k , finally it is working . and thanks for your reply!
122 Views