https://flyte.org logo
#flytekit
Title
# flytekit
h

honnix

01/03/2024, 9:11 AM
Hi. What was the reason we put an upper bound for fsspec https://github.com/flyteorg/flytekit/commit/2a5fa355fdd790f525731b3634a96579e9f744e1#r136117081 ? This setting is now conflicting with other dependencies we have.
fsspec ecosystem versioning is very strict, so if we do something on fsspect itself, we must do the same on other *fs packages. https://github.com/fsspec/gcsfs/issues/584 is what I asked.
k

Kevin Su

01/03/2024, 9:27 AM
I tried to remove that, but some tests are failing
2023.3.0 has some issues
anyway, we should use ! instead of <=
h

honnix

01/03/2024, 10:07 AM
I meant the upper bound btw.
m

MUhwjR9VF3K2QJhL2VvNGoFY

01/03/2024, 6:33 PM
I think someone was working on fixing this: https://github.com/flyteorg/flytekit/pull/1925
k

Kevin Su

01/03/2024, 6:43 PM
oh, nice, thank you, will take a look
h

honnix

01/03/2024, 6:55 PM
Does this make sense to have regardless https://github.com/flyteorg/flytekit/pull/2087 ?
k

Kevin Su

01/03/2024, 6:59 PM
+1, we could remove upper bound later in another PR.
m

MUhwjR9VF3K2QJhL2VvNGoFY

01/08/2024, 9:17 PM
Looks like that MR was closed due to windows issues(?) 😕 It would be nice to to be able to use newer aiobotocore/botocore than what s3fs 2023.9.2 allows.
h

honnix

01/09/2024, 10:53 AM
If there is nothing else preventing us from going beyond the current upper bound, we can for sure remove it. One thing to note is that, pip-compiles legacy resolver has been having hard time resolving a correct match of the fsspec packages family, so internally we always take extra effort pining fsspec packages family to have better backward compatibility. I hope the legacy resolve is not used much anymore as it is going out.
2 Views