https://flyte.org logo
#ask-the-community
Title
# ask-the-community
a

Augie Palacios

05/24/2023, 9:35 PM
is there a way to set postgres and s3/minio passwords as secrets instead of them being in plain text in a deployment?
d

David Espejo (he/him)

05/24/2023, 10:04 PM
HI @Augie Palacios great to hear from you! Is this for
flyte-binary
?
a

Augie Palacios

05/25/2023, 2:20 PM
Hi! You too! was looking in here: https://github.com/flyteorg/flyte/blob/master/charts/flyte/values.yaml right now they are all stored in the k8s
configmap
which can be read in plain text and was wondering if it was currently possible to have them stored in something like
flyte-admin-secrets
?
d

David Espejo (he/him)

05/30/2023, 6:16 PM
Hi @Augie Palacios and sorry for the delay I see here that you could use
additionalVolumes
in
values
to mount and then reference a secret
a

Augie Palacios

05/30/2023, 6:25 PM
hmm ok. I can check on that. How about for setting the db config? If I add
password
to these blocks, they are stored in plain text in the
configmap
for each service. Is there a way to set the postgres password elsewhere without creating a custom secret?
Copy code
# Database configuration
  db:
    datacatalog:
      database:
        port: 5432
        username: postgres
        host: postgres
        dbname: "datacatalog"
    admin:
      database:
        port: 5432
        username: postgres
        host: postgres
        dbname: "flyteadmin"
d

David Espejo (he/him)

05/30/2023, 6:54 PM
you can also use
Copy code
common:
  databaseSecret:
to reference a K8s secret and then one of the templates (
secret.yaml
) will use it
a

Augie Palacios

05/30/2023, 6:57 PM
yes I saw that as well. Does that mean we would have to create the secret file prior to deploying flyte? That would also require creating the
flyte
namespace as well, assuming it was being deployed to the flyte namespace right? If possible we want to avoid any pre or post deployment commands to
helm upgrade --install flyte
d

David Espejo (he/him)

05/30/2023, 7:08 PM
namespace creation can be also on demand adding
--create-namespace
to
helm upgrade
In regards to secrets, in this case they would have to be created beforehand. You could also rely on an external service to handle database passwords dynamically. In this example, a Flyte user shares how they use
ExternalSecrets
operator and AWS Secrets Manager to do it https://github.com/alexifm/flyte-eks-deployment
a

Augie Palacios

05/30/2023, 7:12 PM
Yes that would work for most of our cases but we have a few scenarios where that won't be an option for us and our customers. Our most ideal scenario is the values are put in the flyte
values.yaml
and then end up as secrets via the helm deployment. I was tinkering with the idea of adding this capability but wanted to vet the existing helm charts before I went down that road
g

Gopal Vashishtha

08/07/2023, 6:00 PM
I made a secret called
db-password:
Copy code
kubectl describe secret db-password
Name:         db-password
Namespace:    flyte
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
pass.txt:  32 bytes
and in my values.yaml I have
Copy code
...
  common:
    databaseSecret:
      name: db-password
...
and I confirmed that I can use the password I get from
kubectl
to authenticate in sqladmin:
Copy code
kubectl get secret db-password -o jsonpath='{.data.pass\.txt}' | base64 --decode
but I still see:
Copy code
kubectl logs deploy/flyteadmin -c run-migrations                                
time="2023-08-07T17:55:25Z" level=info msg="Using config file: [/etc/flyte/config/cluster_resources.yaml /etc/flyte/config/db.yaml /etc/flyte/config/domain.yaml /etc/flyte/config/remoteData.yaml /etc/flyte/config/server.yaml /etc/flyte/config/storage.yaml /etc/flyte/config/task_resource_defaults.yaml]"

2023/08/07 17:55:28 /go/pkg/mod/gorm.io/gorm@v1.24.1-0.20221019064659-5dd2bb482755/gorm.go:206
[error] failed to initialize database, got error failed to connect to `host=<http://flyteadmin.postgres.database.usgovcloudapi.net|flyteadmin.postgres.database.usgovcloudapi.net> user=flyteadmin database=flyteadmin`: server error (FATAL: password authentication failed for user "flyteadmin" (SQLSTATE 28P01))
Any idea what I should be doing instead to get the secret mounted correctly?
d

David Espejo (he/him)

08/08/2023, 10:40 AM
@Gopal Vashishtha if you're using
flyte-binary
chart, you can follow the instructions here to leverage a pre created K8s secret
g

Gopal Vashishtha

08/08/2023, 12:01 PM
oh this is cool! I'm using flyte-core though
is that a problem?
92 Views