https://flyte.org logo
#ask-the-community
Title
# ask-the-community
s

sapcode

11/06/2022, 7:50 AM
Dear flyte team, we are about to POC an on-premise full(no sandbox) installation in a local kubernetes cluster. However in the documentation we cant find the sequence of which helm values yamls are supposed to be used when the target is not AWS and not GCP. It would be very helpful to have a section for non cloud installation setup without sandbox. 1) Can someone post an easy installation script for minikube or bare metal kubernetes, without external iDP provider? 2) Is there a build in user management, where we can create users ? 3) is there a default user and password? Some blogs are mentioning foobar. 4) it would be great to have 1 installation yaml for keycloak and the setup values documented which needs to be changed to get it running. Thank you in advance!!!
f

Felix Ruess

11/06/2022, 9:49 AM
@sapcode I don't have an easy installation script, but I just also installed Flyte on my local k3s cluster for testing. I installed via helm and provided my custom override values to configure it. You'll have to take care of some dependencies yourself first though or use the flyte-deps chart. In my case I already had a load balancer, ingress (traefik) and persistent storage setup in my cluster as well as a separate minio cluster for object storage. So I just needed to adapt the postgres manifests from flyte-deps to use a persistentVolumeClaim. And I skipped auth for this test phase...
s

sapcode

11/06/2022, 10:22 AM
Are all components working without authorization setup? What happens if you click the login button in the flyte console UI?
f

Felix Ruess

11/06/2022, 10:31 AM
yes, works with auth disabled. login page is not found then, but you don't need it with auth disabled
If you don't already have an ingress and object store setup, you can pretty much go with the sandbox deployment. If you want to use existing storage and ingress, here is basically what I used after taking care of dependencies:
Copy code
helm repo add flyteorg <https://helm.flyte.org>
helm repo update
helm install flyte flyteorg/flyte-core -n flyte -f values-k3s.yaml
there are a few more things you might need to take care of for GPU, viewing logs, etc
ah, and I forgot to remove
default-pod-template-name: "flyte-nvidia-template"
from these example values, that is my custom pod template which sets the nvidia runtime
s

sapcode

11/06/2022, 11:58 AM
Thank you for all the information i will try this out... Just to clarify with auth disabled there is no user and the flyte console runs in admin mode? Also where did you find the values-k3s.yaml?
For the auth setup We found the following information from: Sören Brunk 10/19/2022, 12:29 PM I found the relevant section to overwrite the default flyteadmin auth server config via helm: configmap: adminServer: server: httpPort: 8088 grpcPort: 8089 dataProxy: upload: storagePrefix: upload security: secure: false useAuth: true allowCors: true allowedOrigins: # Accepting all domains for Sandbox installation - '*' allowedHeaders: - Content-Type auth: appAuth: authServerType: Self selfAuthServer: accessTokenLifespan: 30m0s authorizationCodeLifespan: 5m0s claimSymmetricEncryptionKeySecretName: claim_symmetric_key issuer: "" oldTokenSigningRSAKeySecretName: token_rsa_key_old.pem refreshTokenLifespan: 1h0m0s staticClients: flyte-cli: audience: null grant_types: - refresh_token - authorization_code id: flyte-cli public: true redirect_uris: - <http://localhost:53593/callback> - <http://localhost:12345/callback> response_types: - code - token scopes: - all - offline - access_token flytectl: audience: null grant_types: - refresh_token - authorization_code id: flytectl public: true redirect_uris: - <http://localhost:53593/callback> - <http://localhost:12345/callback> response_types: - code - token scopes: - all - offline - access_token flytepropeller: audience: null client_secret: <your client secret hashed and base64 encoded> grant_types: - refresh_token - client_credentials id: flytepropeller public: false redirect_uris: - <http://localhost:3846/callback> response_types: - token scopes: - all - offline - access_token flyterpropeller.client_secret is the relevant change. Unfortunately, it requires the secret as a base64 encoded hash. I'll try to find how we did that hashing. I think I had to run it through bcrypt: pip install bcrypt python
>> import bcrypt
>> bcrypt.hashpw(b"foobar", bcrypt.gensalt(prefix=b"2a"))
The resulting hash should look something like this:b'$2a$12$d3mGDJwq9F5TiQA1YYm0TOVzvEvcBX5VEw2AW0gqrn7Mvh2InuiCS' then base64 encode it and use that as client_secret in the config. I hope I remember it correctly, I should have documented the steps properly. Perhaps someone from the flyte team familiar with the internal auth server can verify if this is the right way or not. Here we need the matching secret still in cleartext, i.e. secrets: adminOauthClientCredentials: enabled: true clientSecret: foobar clientId: flytepropeller
python -c 'import bcrypt; import base64; print(base64.b64encode(bcrypt.hashpw("mypassword".encode("utf-8"), bcrypt.gensalt(6))))'
Check the sections selfAuthServer and allowedOrigins = * appAuth: thirdPartyConfig: flyteClient: clientId: flytectl redirectUri: <http://localhost:53593/callback> scopes: - offline - all selfAuthServer: staticClients: flyte-cli: id: "flyte-cli" redirect_uris: - "<http://localhost:53593/callback>" - "<http://localhost:12345/callback>" grant_types: - refresh_token - authorization_code response_types: - code - token scopes: - all - offline - access_token public: true flytectl: id: flytectl redirect_uris: - "<http://localhost:53593/callback>" - "<http://localhost:12345/callback>" grant_types: - refresh_token - authorization_code response_types: - code - token scopes: - all - offline - access_token public: true flytepropeller: id: flytepropeller client_secret: JDJhJDA2JGd3N0pNUno1OXpCSzFk43DJkYlUxTHV2MGxRMlFWHNlTkczcElyU3V1TzhZai95ODJsQ2dh redirect_uris: - "<http://localhost:3846/callback>" grant_types: - refresh_token - client_credentials response_types: - token scopes: - all - offline - access_token
f

Felix Ruess

11/06/2022, 8:20 PM
I just put the values-k3s.yaml together myself from looking at the main helm chart values, sandbox values and by finding out what I still needed to change to get it to work with my existing external s3 (minio) storage, especially the config.core.propeller.rawoutput-prefix took me a while to find..
If you also have traefik running as ingress, I can also share my ingress config for that
and task resource limits I just adjusted to my workload
73 Views