https://flyte.org logo
#flytekit
Title
# flytekit
p

Pradithya Aria Pura

12/01/2022, 6:16 AM
Hi All, having issue with recent release of
flyteidl
(1.2.7). I see that the protobuf version is bumped and break compatibility with some python packages.
Copy code
from flytekit.core.context_manager import ExecutionParameters, FlyteContext, FlyteContextManager, FlyteEntities
/usr/local/lib/python3.7/site-packages/flytekit/core/context_manager.py:30: in <module>
    from flytekit.clients import friendly as friendly_client  # noqa
/usr/local/lib/python3.7/site-packages/flytekit/clients/friendly.py:4: in <module>
    from flyteidl.admin import common_pb2 as _common_pb2
/usr/local/lib/python3.7/site-packages/flyteidl/admin/common_pb2.py:5: in <module>
    from google.protobuf.internal import builder as _builder
E   ImportError: cannot import name 'builder' from 'google.protobuf.internal' (/usr/local/lib/python3.7/site-packages/google/protobuf/internal/__init__.py)
Copy code
ERROR: flyteidl 1.2.7 has requirement protobuf<5.0.0,>=4.21.1, but you'll have protobuf 3.15.6 which is incompatible.
k

Ketan (kumare3)

12/01/2022, 6:21 AM
you mean your internal python packages?
p

Pradithya Aria Pura

12/01/2022, 6:21 AM
Ah yes, some requires protobuf <4.21.1
k

Ketan (kumare3)

12/01/2022, 6:23 AM
sorry not sure what we can do about that
protobuf broke some compatibility and we had to migrate
they are deprecating older versions and may hve potential vulnerabilities
also 3.15.6 is around 2 years old
but cc @Eduardo Apolinario (eapolinario) do you have any ideas?
p

Pradithya Aria Pura

12/01/2022, 6:26 AM
also 3.15.6 is around 2 years old
Yes, but it seems they are still maintaining 3.X , last release is 3.20.3 in Sept 2022.
k

Ketan (kumare3)

12/01/2022, 6:26 AM
interesting
i think eduardo might know why we upgraded
p

Pradithya Aria Pura

12/01/2022, 6:27 AM
Personally I’d like to support 4.X since there is huge performance improvement.
Temporarily we can pin flyteidl to 1.2.5 and hopefully does the trick
e

Eduardo Apolinario (eapolinario)

12/01/2022, 6:16 PM
@Pradithya Aria Pura, sorry for the breakage. We're working to bring in the new versions of protobuf and also grcp python libraries, but flyteidl 1.2.7 is in a bad state. I just yanked it from pypi. Expect a new version (this time will be 1.3.0) which will allow us a clean break (since flytekit pins the flyteidl version). Again, sorry for the breakage and we will have this fixed asap.
p

Pradithya Aria Pura

12/02/2022, 1:56 AM
It’s ok. One question, will the next release of flyte backend be compatible with the current flytekit and flyteidl? In our side, we might need to pin the flytekit/flyteidl version temporarily until we can test the compatibility with newer protobuf, but at the same time still can upgrade the flyte backend.
k

Ketan (kumare3)

12/02/2022, 5:27 AM
I don’t think python protobuf version affects wire format
It’s purely code changes
p

Pradithya Aria Pura

12/02/2022, 5:28 AM
Got it, thanks for confirmation!
e

Eduardo Apolinario (eapolinario)

12/02/2022, 5:29 AM
that's correct. The main change is in the library used to marshal / unmarshal bytes (in other words, no change to the wire format as Ketan mentioned)
76 Views