Hi everyone, I'm desperately trying to setup flyte-core with an s3 bucket and provide my access key and secret key via a secret. I can't find how to do that, the documentation isn't clear on what form should that secret take and the ai bot ins't helping and giving contradictory and false information. Can someone please provide an example? Thanks a lot

If you use a secret you gotta pass the whole storage config in it

ok that's what I thought thank you

ie.

apiVersion: <http://external-secrets.io/v1beta1|external-secrets.io/v1beta1>
kind: ExternalSecret
metadata:
  name: metadata-storage-config
  namespace: "{{ .Release.Namespace }}"
spec:
  refreshInterval: "0"
  secretStoreRef:
    kind: SecretStore
    name: vault
  target:
    name: metadata-storage-config
    template:
      engineVersion: v2
      data:
        storage.yaml: |
          storage:
            container: flyte-metadata-prod
            stow:
              config:
                access_key_id: "{{ `{{ .access_key_id }}` }}"
                authType: accesskey
                disable_ssl: false
                endpoint: ....
                region: us-east-1
                secret_key: "{{ `{{ .secret_key }}` }}"
              kind: s3
            type: stow
            enable-multicontainer: false

We use the external secrets operator to render a secret for us

Thank you @clean-glass-36808 We use external secrets and vault as well. I am using a different configuration:

type: s3    
container: flyte
connection:
  auth-type:   accesskey
  access-key:  {{ .accessKey }}
  secret-Key:  {{ .secretKey }}
  ...

Does that make any difference (I'm guessint your doesn't generate the stow warning described above)? Also when running the simple hello world workflow remotely I'm getting this execution error:

File "/usr/local/lib/python3.12/site-packages/flytekit/core/data_persistence.py", line 614, in async_get_data
    raise FlyteDownloadDataException(
flytekit.exceptions.system.FlyteDownloadDataException: SYSTEM:DownloadDataError: error=Failed to get data from <s3://flyte/test/development/5LK6LWBPEDH3AH3S4YVSN67YFY======/fastd041ba9d98688a26eb97465bc514f929.tar.gz> to ./ (recursive=False).

How are credentials supposed to be managed during task execution? Thanks

Not sure about the stow warning. Ah yeah this isn't documented very well and probably should be but you need to separately plumb credentials to be available for the flyte tasks where they run. We use a pod template to inject some basic environment variables for us. Let me dig up another thread

Oh thank you. This is unfortunate cause we want to run tasks in user owned namespaces and don't want to share the bucket credentials with users. Would you have any recommendation?

I'm just a user so I can only talk about what we do. We have namespaces per project/domain combo. And then we have different S3 credentials that we install into each of those namespaces, scoped to the specific project/domain. Our users don't have access to the cluster so its not an issue for us.

Ah ok I see. We're building an internal platform, so our users have access to their tenant namespaces. I would say it's not possible to setup flyte for true multi-tenancy for now, and we haven't gone into the RBAC/authz of the console/admin. Thanks for you kind help anyway @clean-glass-36808 !

I built RBAC/authz for flyte but it was not merged. Other folks have used it with success though: https://github.com/flyteorg/flyte/pull/6190