Hey folks so I m exploring how to set up the CI CD pipeline

Question

Hey folks so I m exploring how to set up the CI CD pipeline now that I have some workflows running locally In order to do this I was experimenting with the flyte sandbox docker image to register the new version of the workflows and so on I m running into an error though `flytectl get workflows admin endpoint lt remote host gt 30080 p lt project name gt d development` is successful `docker run rm entrypoint flytectl <http cr flyte org flyteorg flyte sandbox|cr flyte org flyteorg flyte sandbox> get workflows admin endpoint lt remote host gt 30080 p lt project name gt d development` throws an error ```time= 2022 06 27T20 18 25Z level=info msg= 0 Couldn t find a config file Relying on env vars and pflags json level error msg failed to initialize token source provider Err failed to fetch auth metadata Error rpc error code = Unavailable desc = connection error desc = transport authentication handshake failed tls first record does not look like a TLS handshake ts 2022 06 27T20 18 26Z json level warning msg Starting an unauthenticated client because can t create authenticated channel without a TokenSourceProvider ts 2022 06 27T20 18 26Z json level info msg Initialized Admin client ts 2022 06 27T20 18 26Z Error rpc error code = Unavailable desc = connection error desc = transport authentication handshake failed tls first record does not look like a TLS handshake json level error msg rpc error code = Unavailable desc = connection error desc = transport authentication handshake failed tls first record does not look like a TLS handshake ts 2022 06 27T20 18 27Z ```

Yuvraj · Answer

Your local system have flytectl config and Sandbox config use `insecure` connection but when you run same flytectl command inside the sandbox image then you don t have flytectl config inside the container and flytectl use secure connection For fixing it you can pass ` admin insecure` flag it will disable the tls

Rupsha Chaudhuri · Answer

thank you let me try that

Rupsha Chaudhuri · Answer

should I be using the sandbox for this purpose or is there a different docker image which has flytectl etc for me to use in the CI CD pipeline

Yuvraj · Answer

no you can run flytectl from your host machine until you have access of flyteadmin server

Yuvraj · Answer

We have github action for setup and registry workflow ```steps uses actions checkout v2 name Setup flytectl uses unionai flytectl setup action v0 0 1 with version 0 1 8 uses unionai flyte register action v0 0 1 with version 0 1 8 The version of workflow proto project flytesnacks domain development archive true```

Yuvraj · Answer

You can also check our end2end test pipeline workflow <https github com flyteorg flytetools blob master github workflows end2end yml> You can use any remote cluster in place of sandbox You just need right flytectl config

Katrina P · Answer

Regarding the GHA scripts shared how do you handle auth access to your cluster where flyte is deployed

Yuvraj · Answer

By default flytectl use pkce for authentication but you can use clientid amp secret for example ```admin For GRPC endpoints you might want to use dns flyte myexample com endpoint dns flyte org Change insecure flag to ensure that you use the right setting for your environment insecure false clientId authType ClientSecret clientSecretLocation home runner secret location logger Logger settings to control logger output Useful to debug logger show source true level 1 ```

Yuvraj · Answer

Flytectl also has flags for passing these values ``` admin authType string Type of OAuth2 flow used for communicating with admin ClientSecret Pkce ExternalCommand are valid values default ClientSecret admin authorizationHeader string Custom metadata header to pass JWT admin authorizationServerUrl string This is the URL to your IdP s authorization server It ll default to Endpoint admin caCertFilePath string Use specified certificate file to verify the admin server peer admin clientId string Client ID default flytepropeller admin clientSecretLocation string File containing the client secret default etc secrets client secret admin command strings Command for external authentication token generation admin endpoint string For admin types specify where the uri of the service is located admin insecure Use insecure connection admin insecureSkipVerify InsecureSkipVerify controls whether a client verifies the server s certificate chain and host name Caution shouldn t be use for production usecases admin maxBackoffDelay string Max delay for grpc backoff default 8s admin maxRetries int Max number of gRPC retries default 4 admin perRetryTimeout string gRPC per retry timeout default 15s admin pkceConfig refreshTime string default 5m0s admin pkceConfig timeout string default 15s admin scopes strings List of scopes to request admin tokenRefreshWindow string Max duration between token refresh attempt and token expiry default 0s admin tokenUrl string OPTIONAL Your IdP s token endpoint It ll be discovered from flyte admin s OAuth Metadata endpoint if not provided admin useAuth Deprecated Auth will be enabled disabled based on admin s dynamically discovered information ```