Hey team! One question here. Flytekit latest versions use fsspec under the hood for interacting with remote storages AFAIK. Do we need to manually install the underlying driver for this to work? For example,

s3fs

or

gcsfs

, or should it work transparently? One of our projects is having issues due to this, and I’m not sure if it’s a problem on the project’s dependencies, or something that we need to specifically install besides flytekit. Thanks!

My understanding is that those are installed per default in the flyte docker images. In case you have own docker image you would need to install them.

Currently they are installed by default, but we are thinking of making them extras as they bloat flytekit

Weirdly, we were having some access denied errors that were fixed after upgrading

s3fs

specifically. But might be something specific. If it happens to someone else here, let me know and I can help taking a look 👀

Thank you, one reason I guy be a very old version of s3fs, is e should make sure that is infact not happening

Yep, that was the case for us

We should make s3fs greater than some version, as eks Iam roles only work after a specific boot version. Cc @Yee I see it is a free floating dependency, it should have a lower bound