https://flyte.org logo
Join Slack
Powered by
Hey folks.. so I'm exploring how to set up the CI/...
# flyte-support
l
Hey folks.. so I'm exploring how to set up the CI/CD pipeline now that I have some workflows running locally. In order to do this I was experimenting with the flyte-sandbox docker image to register the new version of the workflows and so on. I'm running into an error though 
flytectl get workflows --admin.endpoint <remote_host>:30080 -p <project_name> -d development
is successful 
docker run --rm --entrypoint flytectl <http://cr.flyte.org/flyteorg/flyte-sandbox|cr.flyte.org/flyteorg/flyte-sandbox> get workflows --admin.endpoint <remote_host>:30080 -p <project_name> -d development
throws an error
Copy code
time="2022-06-27T20:18:25Z" level=info msg="[0] Couldn't find a config file []. Relying on env vars and pflags."
{"json":{},"level":"error","msg":"failed to initialize token source provider. Err: failed to fetch auth metadata. Error: rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: first record does not look like a TLS handshake\"","ts":"2022-06-27T20:18:26Z"}
{"json":{},"level":"warning","msg":"Starting an unauthenticated client because: can't create authenticated channel without a TokenSourceProvider","ts":"2022-06-27T20:18:26Z"}
{"json":{},"level":"info","msg":"Initialized Admin client","ts":"2022-06-27T20:18:26Z"}
Error: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"
{"json":{},"level":"error","msg":"rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: first record does not look like a TLS handshake\"","ts":"2022-06-27T20:18:27Z"}
g
Your local system have flytectl config and Sandbox config use 
insecure
connection but when you run same flytectl command inside the sandbox image then you don’t have flytectl config inside the container and flytectl use secure connection. For fixing it you can pass 
--admin.insecure
flag it will disable the tls.
l
thank you.. let me try that
should I be using the sandbox for this purpose? or is there a different docker image which has flytectl etc for me to use in the CI/CD pipeline
g
no you can run flytectl from your host machine until you have access of flyteadmin server
We have github action for setup and registry workflow • https://github.com/unionai-oss/flyte-register-action • https://github.com/unionai-oss/flytectl-setup-action
Copy code
steps:
  - uses: actions/checkout@v2
  - name: Setup flytectl
    uses: unionai/flytectl-setup-action@v0.0.1
    with:
      version: "0.1.8"
  - uses: unionai/flyte-register-action@v0.0.1
    with:
      version: '0.1.8' # The version of workflow
      proto: '<https://github.com/flyteorg/flytesnacks/releases/download/v0.2.89/flytesnacks-core.tgz>'
      project: 'flytesnacks'
      domain: 'development'
      archive: true
You can also check our end2end test pipeline workflow https://github.com/flyteorg/flytetools/blob/master/.github/workflows/end2end.yml, You can use any remote cluster in place of sandbox. You just need right flytectl config
l
Regarding the GHA scripts shared, how do you handle auth/access to your cluster where flyte is deployed?
g
By default flytectl use pkce for authentication but you can use clientid & secret. for example
Copy code
admin:
  # For GRPC endpoints you might want to use dns:///flyte.myexample.com
  endpoint: dns:///flyte.org
  # Change insecure flag to ensure that you use the right setting for your environment
  insecure: false
  clientId: *********
  authType: ClientSecret
  clientSecretLocation: /home/runner/secret_location
logger:
  # Logger settings to control logger output. Useful to debug logger:
  show-source: true
  level: 1
Flytectl also has flags for passing these values
Copy code
--admin.authType string                      Type of OAuth2 flow used for communicating with admin.ClientSecret, Pkce, ExternalCommand are valid values (default "ClientSecret")
      --admin.authorizationHeader string           Custom metadata header to pass JWT
      --admin.authorizationServerUrl string        This is the URL to your IdP's authorization server. It'll default to Endpoint
      --admin.caCertFilePath string                Use specified certificate file to verify the admin server peer.
      --admin.clientId string                      Client ID (default "flytepropeller")
      --admin.clientSecretLocation string          File containing the client secret (default "/etc/secrets/client_secret")
      --admin.command strings                      Command for external authentication token generation
      --admin.endpoint string                      For admin types,  specify where the uri of the service is located.
      --admin.insecure                             Use insecure connection.
      --admin.insecureSkipVerify                   InsecureSkipVerify controls whether a client verifies the server's certificate chain and host name. Caution : shouldn't be use for production usecases'
      --admin.maxBackoffDelay string               Max delay for grpc backoff (default "8s")
      --admin.maxRetries int                       Max number of gRPC retries (default 4)
      --admin.perRetryTimeout string               gRPC per retry timeout (default "15s")
      --admin.pkceConfig.refreshTime string         (default "5m0s")
      --admin.pkceConfig.timeout string             (default "15s")
      --admin.scopes strings                       List of scopes to request
      --admin.tokenRefreshWindow string            Max duration between token refresh attempt and token expiry. (default "0s")
      --admin.tokenUrl string                      OPTIONAL: Your IdP's token endpoint. It'll be discovered from flyte admin's OAuth Metadata endpoint if not provided.
      --admin.useAuth                              Deprecated: Auth will be enabled/disabled based on admin's dynamically discovered information.
👍 1
216 Views
Open in Slack
PreviousNext
Powered by