Rahul Mehta
10/31/2022, 4:40 AMPickleTransformer
passed as args to the LaunchPlan 🧵S3Persistence
is implemented using subprocess calls to awscli vs using boto3 to directly interact with S3? It appears that the version of awscli in our github actions runner image expects python 3.8, but python 3.9 is the only available distribution in the environment. Seems like this could be sidestepped entirely by avoiding the subprocess calls?Samhita Alla
Ketan (kumare3)
Rahul Mehta
10/31/2022, 6:23 AMKetan (kumare3)
Rahul Mehta
10/31/2022, 6:24 AMKetan (kumare3)
Samhita Alla
flytekitplugins-data-fsspec[aws]
requires flytekitplugins-data-fsspec
and s3fs>=2021.7.0
libraries.Rahul Mehta
10/31/2022, 2:03 PM