we recently upgraded our flyte binary in EKS to `1...
# ask-the-community
we recently upgraded our flyte binary in EKS to
. we started the process of updating our docker images to use the latest
as well with:
Copy code
RUN pip install boto3 awscli flytekit==1.7.0
but we’re getting conflicting dependencies related to `fsspec`:
Copy code
RROR: Cannot install flytekit and flytekit==1.7.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    flytekit 1.7.0 depends on fsspec>=2023.3.0
    s3fs 2023.1.0 depends on fsspec==2023.1.0
this works if we downgraded back to
cc @Eduardo Apolinario (eapolinario)
flytekit 1.7 uses s3fs, so you don’t need to install boto3 and awscli
@Brian Tang, do you have a specific use of boto3 and/or awscli in your flytekit images? Since flytekit 1.5 we moved the data persistence layer to rely solely on fsspec (which essentially means that we don't use boto3 nor awscli anymore as part of regular flytekit operations).
we need boto3 in the container image, as we’re interacting w dynamodb and s3 on our application layer. as for awscli, I did not know it’s no longer required after 1.5 - it’s there because it was previously a requirement for flyte container images
@Eduardo Apolinario (eapolinario) - i’ve tried removing
, i.e.
pip install boto3 flytekit==1.7.0
but still getting the deps conflict. we need
as it’s part of our application layer to interact w our aws infra
we'll get back to you, @Brian Tang. Discussing this internally at the moment.
@Brian Tang, this conflict is caused by incompatible versions of botocore. Can you pin either
ok will test and get back to you, thanks Eduardo
Was this ever resolved? We have a need for Boto3 in our workflows and I can't seem to find a combination of versions of boto3 and flytekit that work. Downgrading Flytekit to 1.5.0 allows me to install boto3 but when I try to run pyflyte to register the workflow I get an errors, then when I upgrade Flytekit to a more recent version it works. Any assistance/suggestions would be greatly appreciated. Thank you