Hi All, having issue with recent release of `flyte...
# flytekit
m
Hi All, having issue with recent release of
flyteidl
(1.2.7). I see that the protobuf version is bumped and break compatibility with some python packages.
Copy code
from flytekit.core.context_manager import ExecutionParameters, FlyteContext, FlyteContextManager, FlyteEntities
/usr/local/lib/python3.7/site-packages/flytekit/core/context_manager.py:30: in <module>
    from flytekit.clients import friendly as friendly_client  # noqa
/usr/local/lib/python3.7/site-packages/flytekit/clients/friendly.py:4: in <module>
    from flyteidl.admin import common_pb2 as _common_pb2
/usr/local/lib/python3.7/site-packages/flyteidl/admin/common_pb2.py:5: in <module>
    from google.protobuf.internal import builder as _builder
E   ImportError: cannot import name 'builder' from 'google.protobuf.internal' (/usr/local/lib/python3.7/site-packages/google/protobuf/internal/__init__.py)
Copy code
ERROR: flyteidl 1.2.7 has requirement protobuf<5.0.0,>=4.21.1, but you'll have protobuf 3.15.6 which is incompatible.
f
you mean your internal python packages?
m
Ah yes, some requires protobuf <4.21.1
f
sorry not sure what we can do about that
👍 1
protobuf broke some compatibility and we had to migrate
they are deprecating older versions and may hve potential vulnerabilities
also 3.15.6 is around 2 years old
but cc @high-accountant-32689 do you have any ideas?
m
also 3.15.6 is around 2 years old
Yes, but it seems they are still maintaining 3.X , last release is 3.20.3 in Sept 2022.
f
interesting
i think eduardo might know why we upgraded
m
Personally I’d like to support 4.X since there is huge performance improvement.
Temporarily we can pin flyteidl to 1.2.5 and hopefully does the trick
h
@most-gold-65483, sorry for the breakage. We're working to bring in the new versions of protobuf and also grcp python libraries, but flyteidl 1.2.7 is in a bad state. I just yanked it from pypi. Expect a new version (this time will be 1.3.0) which will allow us a clean break (since flytekit pins the flyteidl version). Again, sorry for the breakage and we will have this fixed asap.
m
It’s ok. One question, will the next release of flyte backend be compatible with the current flytekit and flyteidl? In our side, we might need to pin the flytekit/flyteidl version temporarily until we can test the compatibility with newer protobuf, but at the same time still can upgrade the flyte backend.
f
I don’t think python protobuf version affects wire format
🙏 1
It’s purely code changes
m
Got it, thanks for confirmation!
h
that's correct. The main change is in the library used to marshal / unmarshal bytes (in other words, no change to the wire format as Ketan mentioned)
👍 1
158 Views