Is there a particular reason

S3Persistence

is implemented using subprocess calls to awscli vs using boto3 to directly interact with S3? It appears that the version of awscli in our github actions runner image expects python 3.8, but python 3.9 is the only available distribution in the environment. Seems like this could be sidestepped entirely by avoiding the subprocess calls?

You can install it using the data persistence plugins and we are thinking of making it default - and not breaking folks will be hard - cc @Eduardo Apolinario (eapolinario) / @Yee

@Samhita Alla can you point to docs of how to install flytekitplugins-data-fsspec[aws]

Would you be open to helping

let’s bring this up In a townhall

We are actually thinking of writing a faster layer