https://flyte.org logo
#ask-the-community
Title
# ask-the-community
m

Marti Jorda Roca

10/27/2023, 11:54 AM
🤕 Issue: Use AWS Secrets Manager secrets in Flyte-core chart 🤕 Hi, our team is trying to use the AWS Secrets and Configuration Provider (ASCP) for the Kubernetes Secrets Store CSI Driver for our secrets in the
flyte-core
chart. We have defined our chart values as follow:
Copy code
databaseSecret:
    name: db-pass
    volume:
      - name: db-pass
        csi:
          driver: secrets-store.csi.k8s.io
          readOnly: true
          volumeAttributes:
            secretProviderClass: aws-db-secrets
    secretManifest:
      apiVersion: secrets-store.csi.x-k8s.io/v1
      kind: SecretProviderClass
      metadata:
        name: aws-db-secrets
      spec:
        provider: aws
        parameters:
          objects: |
            - objectName: "<our secret arn>
              jmesPath: 
                  - path: password
                    objectAlias: dbpassword
But the
_helpers.tpl
overrides the helm template with a volumes value that is not the required for AWS to work. How we can avoid that? Do we need to change the
_helpers.tpl
to add this option? Thank you 🫶
s

Samhita Alla

10/30/2023, 4:51 AM
I believe so. Is it causing any failures in your deployment?
m

Marti Jorda Roca

10/30/2023, 10:05 AM
yes it does not find my secret
Copy code
Warning  FailedMount  26m (x371 over 3d)     kubelet  Unable to attach or mount volumes: unmounted volumes=[db-pass], unattached volumes=[kube-api-access-jgj6s aws-iam-token db-pass config-volume]: timed out waiting for the condition
the volume pod should be like this to work wiht aws secretes manager
Copy code
volumes:
      - name: db-pass
        csi:
          driver: secrets-store.csi.k8s.io
          readOnly: true
          volumeAttributes:
            secretProviderClass: aws-db-secrets
but it appears automatically like this for the _helpers.tpl
Copy code
volumes:
      - name: db-pass
        secret:
          secretName: db-pass
how we can override this default secret configuration?
s

Samhita Alla

10/30/2023, 1:35 PM
@David Espejo (he/him), any idea how to fix this?
m

Marti Jorda Roca

10/30/2023, 3:41 PM
Hi I solved my problem with this configuration:
Copy code
databaseSecret:
    name: db-pass
    secretManifest:
      apiVersion: <http://secrets-store.csi.x-k8s.io/v1|secrets-store.csi.x-k8s.io/v1>
      kind: SecretProviderClass
      metadata:
        name: aws-db-secrets-spc
      spec:
        provider: aws
        parameters:
          objects: |
            - objectName: "{{ .Values.userSettings.db_secret }}"
              objectType: "secretsmanager"
              jmesPath: 
                  - path: password
                    objectAlias: dbpassword
        # Create k8s secret. It requires volume mount first in the pod and then sync.
        secretObjects:
          - secretName: db-pass
            type: Opaque
            data:
              - objectName: dbpassword
                key: dbpassword
With this the aws csi driver creats a base k8s secret and works 🤟
d

David Espejo (he/him)

10/30/2023, 3:59 PM
Thanks for sharing @Marti Jorda Roca I was just starting to play with this 🙂 So, the
dbpassword
secret ends up mounted as the
db-pass
volume on
flyteadmin
right?
m

Marti Jorda Roca

10/30/2023, 4:12 PM
yes this is it
sry and also it is required to create another volume, more info
Copy code
datacatalog:

  # mount another db secret to activate secret-store-csi to create db-pass k8s secret.
  additionalVolumes:
    - name: aws-secret
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: aws-db-secrets-spc

  additionalVolumeMounts:
    - name: aws-secret
      mountPath: "/mnt/aws-secrets"
      readOnly: true
its a little bit hacky
the best would be to allow to override
databaseSecret.volume
d

David Espejo (he/him)

10/30/2023, 9:12 PM
@Marti Jorda Roca agree. Would you be up to create an Issue to capture that request?
m

Marti Jorda Roca

10/31/2023, 11:31 AM
hi, yes I have open this issue. If I need to change anything just ask for it ( it is my first issue 😅)
d

David Espejo (he/him)

10/31/2023, 3:28 PM
Thank you @Marti Jorda Roca What are your thoughts on using this capability?: https://github.com/flyteorg/flyte/pull/3807 I know it's not the same but in the meantime, is an option to avoid plain text DB password on values nvm, you're using
flyte-core
2 Views