Hey Flyte folks! 🙂 I am currently trying to get Flyte to work with Azure. I successfully configured "stow" config for admin and propeller and also managed to do an upload (fast-register) using an

abfs://

prefixed url. However, propeller produces a url like this for the pyflyte command:

pyflyte-execute --inputs <afs://flyte-data/metadata/propeller/test-azure-development-boygblti4g/n3/data/0/n0/inputs.pb>

which lead to the following error in flytekit:

Original exception: No plugin found for matching protocol of path <afs://flyte-data/metadata/propeller/test-azure-development-boygblti4g/n2/data/0/n0/0>

I believe the value

afs

is taken from https://github.com/graymeta/stow - however I could not identify the place in propeller yet. Did I miss how to configure this properly?

hi @Stephan Gref this is amazing - thank you so much. We are extremely excited about this work. Full community support from us

Sure, I use these versions:

adlfs = "2022.2.0"
flytekit = "^0.30.0"
flytekitplugins-data-fsspec = "^0.30.0"

ok awesome, and it seems fsspec prefers abfs

Uploading an

abfs

prefixed blob works - but

afs

seems not supported.

ya makes sense

I did not find out where

afs

is coming from exactly

ya I do not think the flytebacked constructs the

afs

prefix. We use flytestdlib - storage

ah yes - there it is: azure.Kind: func(bucket string) DataReference {

ya

but we can easily change this

that would be amazing 🙂

actually this is just in stdlib - ohh wow this is trivial.

I will do so.

no