Hello! Back with more Azure adventures 🙂 I've managed to trigger a basic workflow from

flytesnacks

now, but in the UI I see the following error for my execution:

FlyteAssertion: Failed to get data from 
<abfs://flyte/flytesnacks/development/A46TBP35TISQ2EVJVNOGLJTARM======/script_mod>
e.tar.gz to /root/ (recursive=False).

Original exception: unable to connect to account for Must provide either a 
connection_string or account_name with credentials!!

Here's my storage config in the helm chart:

storage:
  type: custom
  bucketName: "flyte"
  custom:
    container: "flyte"
    enable-multicontainer: true
    connection: {}
    type: stow
    stow:
      kind: azure
      config:
        account: "<storage_account_name>"
        key: "<storage_account_key>"
  limits:
    maxDownloadMBs: 10

Does anyone know what would cause this? My first intuition would be that this URI is wrong:

<abfs://flyte/flytesnacks/development/A46TBP35TISQ2EVJVNOGLJTARM======/script_mode.tar.gz>

because in Azure the path should look like this:

abfs://<CONTAINER>@<STORAGE_ACCOUNT>.<http://dfs.core.windows.net/flytesnacks/development/A46TBP35TISQ2EVJVNOGLJTARM======/script_mode.tar.gz|dfs.core.windows.net/flytesnacks/development/A46TBP35TISQ2EVJVNOGLJTARM======/script_mode.tar.gz>

does you pod have IAM role to access data in abfs?

Yes this is delegated to fsspec. But we have heard fsspec azure file system has some bugs from some community folks but it was fixed in a later version

Could be an issue with my pod’s role yeah 🤔 . But why would I be able to create the object and not retrieve it? Looking at the fsspec code, it seems like it cannot find my credentials. I’ll double check and report back!

Victor I was able to resolve this issue by setting an environment variable in my pod template:

env:
          - name: AZURE_STORAGE_CONNECTION_STRING
            valueFrom:
              secretKeyRef:
                name: flyteruntimesecret

is this because Azure does not have something like IAM roles for Service accounts or Workload identity?

Oh nice! For which pod did you do that? I tried it earlier but you can only pass extra env variables to flyteadmin in the helm chart, and I think my error is in propeller

Omg azure @calm-zoo-68637 and other can someone please help us write a working azure override

Will definitely write something once I have a working setup

To dust off this thread a bit, I also bumped into this issue when testing in connection with flyteorg/stow/pull/9. I was pointed here by Samhita (thanks for that, btw). I can add a little more context to the situation. While adding the

AZURE_STORAGE_CONNECTION_STRING

env var to all task pods will definitely work, it's not the only path. Adding

AZURE_STORAGE_ACCOUNT_NAME

to all pods will also work, as long as the credentials can be loaded from the usual spots which you can find here.

hi guys, I have faced the same issue described above. I have kubernetes cluster in Azure cloud, now I have created storage account and PostgresSQL, everything have started successfully, but during executing workflow I am getting such errors in logs, ---------------------- FlyteAssertion: Failed to get data from abfs://my-flyte-container/flytesnacks/development/I62Z4WQMINA7777777777RU==== ==/script_mode.tar.gz to /root/ (recursive=False). Original exception: unable to connect to account for Must provide either a connection_string or account_name with credentials!! ----------------------- I have edited the deployment (flyte-backend-flyte-binary) and manually added the ENV variable to the config for temporary testing, but no success spec: containers: - args: - start - --config - /etc/flyte/config.d/*.yaml env: - name: POD_NAME valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.namespace - name: AZURE_STORAGE_CONNECTION_STRING value: abfs://my-flyte-container@flyte.dfs.core.windows.net/flytesnacks/development/I62Z4WQMINAWG7777777RU======/script_mode.tar.gz Could you advice, where am I wrong?