https://flyte.org logo
Join Slack
Powered by
Hi all <#CP2HDHKE1|ask-the-community> I have set u...
# flyte-on-gcp
j
Hi all #ask-the-community I have set up a Flyte in GCP with Google’s OAuth2 Client Credentials successfully. I am able to access everything. It just asks for your google account as expected, you click your google account and it lets you in as expected. This is the config normal users use:
Copy code
admin:
  endpoint: dns:///flyte.somedomain.com
  insecure: false
logger:
 show-source: true
 level: 0
storage:
  type: stow
  stow:
    kind: google
    config:
      json: ""
      project_id: brain-project # GCP Project ID
      scopes: <https://www.googleapis.com/auth/devstorage.read_write>
  container: bucket-flyte
Now, I want to trigger a workflow from a backend that is running inside the same cluster as Flyte, but I can’t figure out how I can get 
flytectl
to connect and trigger the workflow. Among other things, I tried several flyte configs, but I always get some type of error. E.g.
Copy code
Response: {"error":"invalid_client","error_description":"Client authentication failed (e.g., unknown client, no client authentication included, or unsupported authentication method)."}
Does anyone know how I could achieve this? What config could I use? Any help is much appreciated 🙏
d
Hi @Joaquín Rives Gambín. Great to have you here. So, what Helm chart did you use to deploy Flyte? Was it 
flyte-binary
or 
flyte-core
?
j
Hi! I used 
flyte-core
. I followed this instructions: GCP (GKE) Setup
m
This looks like the error when there is no browser agent present. The default oauth flow requires a browser. To do headless authentication from within your cluster, you may want to use the 
ClientSecret
https://docs.flyte.org/en/latest/deployment/configuration/auth_setup.html#continuous-integration-ci
d
@Mark Waylonis or 
DeviceFlow
?
m
Yeah if its a user interacting with flytectl then device flow would seem like a good fit. If its an automated (non-interactive) process, then client secret would be the best fit according to my understanding
j
🎉 Yeah, I just did it like this and it worked!
Copy code
flytectl --admin.endpoint=dns:///flyte.somedomain.com --admin.clientId=...<http://apps.googleusercontent.com|apps.googleusercontent.com> --admin.clientSecretLocation=/home/joaquin/.flyte/client_secret --admin.insecure=false get workflows -p flytesnacks -d development
Not sure why it wasn't working earlier when I tried the same but on the flyte 
config.yaml
, probably I missed something. Thanks a lot David and Mark, you did me a big favor!
@Mark Waylonis @David Espejo (he/him) Well, actually still doesn't work 😅 It worked because I had logged in before and it had the token stored somehow 😢 It still shows the same error:
Copy code
ection Info: [Endpoint: dns:///silosnap.silobrain.com, InsecureConnection?: false, AuthMode: ClientSecret]: rpc error: code = Unauthenticated desc = transport: per-RPC creds failed due to error: failed to get token: oauth2: cannot fetch token: 401 Unauthorized
Response: {"error":"invalid_client","error_description":"Client authentication failed (e.g., unknown client, no client authentication included, or unsupported authentication method)."}
{"json":{},"level":"error","msg":"Connection Info: [Endpoint: dns:///silosnap.silobrain.com, InsecureConnection?: false, AuthMode: ClientSecret]: rpc error: code = Unauthenticated desc = transport: per-RPC creds failed due to error: failed to get token: oauth2: cannot fetch token: 401 Unauthorized\nResponse: {\"error\":\"invalid_client\",\"error_description\":\"Client authentication failed (e.g., unknown client, no client authentication included, or unsupported authentication method).\"}","ts":"2023-10-06T08:24:46+03:00"}
Only works after logging in the browser:
Could it be something related with my my credentials (Client ID for Web application) configuration?
Found a work around using the 
flytepropeller
client_id and secret. I wonder if I should have added the GCP Auth credentials here in the helm values when deploying:
Copy code
secrets:
  adminOauthClientCredentials:
 # -- If enabled is true, helm will create and manage `flyte-secret-auth` and populate it with `clientSecret`.
 # If enabled is false, it's up to the user to create `flyte-secret-auth`
    enabled: true
  # Use the non-encoded version of the random password
    clientSecret: <secret>
    clientId: flytepropeller
d
hey @Joaquín Rives Gambín I feel that the flow you need is 
PKCE
instead of 
ClientSecret
, especially if you expect that a 
pyflyte
invocation prompts an auth screen on the browser. From what I read, it seems like you plan to use Google only for Identity (OIDC) but keep using Flyte's internal auth server?
j
@David Espejo (he/him) Yes, that part is working correctly for the users using 
PKCE
, they get prompted to log in and can access correctly. The problem is that we want to trigger pipelines automatically from a nodejs backend, so no prompt screen possible here.
s
Hey @Joaquín Rives Gambín, did you manage to find a solution? We are facing the same issue.
j
@Shahil Mawjee Oh sorry, for the late response, I missed the notification. No, unfortunately didn't found any solution. We are still using the existing 
flytepropeller
client_id and secret as a work around:
Copy code
# flyte-config.yaml
admin:
  endpoint: dns:///xxxx.com
  clientId: flytepropeller
  clientSecretLocation: /usr/api/app/keys/client_secret  # file containing the secret
  insecure: false
The 
flytepropeller
secret is the one created when deploying flyte following this instructions: https://docs.flyte.org/en/latest/deployment/configuration/auth_setup.html
With that config and secret I am able to trigger workflows from the backend with flytectl, which is enough for us at the moment.
s
Thanks @Joaquín Rives Gambín
14 Views
Open in Slack
PreviousNext
Powered by