Any recommendations for python dependencies that d...
# ask-the-community
Any recommendations for python dependencies that don’t work with
dependencies? I’m trying to use
in a pipeline task, but with
, which are the most up to date, supposedly compatible versions for these, using
, I’m getting an error when I import
Copy code
TypeError: Descriptors cannot not be created directly.
If this call came from a file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
However, It doesn’t look like I can lower the
version due to
dependencies, and the other option doesn’t seem to be working, but I’m seeing if I can get it to work. Is this something I should submit a ticket for, to try to allow for better compatibility? Usually we use
or greater, which doesn’t work at all, but would be great to have compatibility for
protobuf package is not backwards compatible, so one has to to decide to use version >=4.0 or <4.0. flytekit and most other packages have upgraded. So it seems like you are going to have to wait for or do the work of updating protobuf in coremltools :/
You could also try using very old flytekit back when it used protobuf <4 but I am not sure how workable that is.
Ok, that’s good to know, thanks. I’m guessing its impossible to run a task without flytekit installed?
You could have a task with flytekit that shells out to a different venv without flytekit maybe? It would be messy and you would be giving up a lot of flyte functionality.
Interesting, yeah I see what you’re saying. It could maybe work as a workaround until
Though I’m not sure on exactly what that would look like
I think your “best” option is probably to just fork coremltools and build your own wheels with updated protobuf if you want to use it with flyte (or pretty much anything else at this point)
Sounds good, thanks for the help
you can use flytekit<1.3, that is the latest proto3 supported flytekit which my team are still using till date with flyte 1.10 server. You would miss out on features from 1.3 onwards on the downside
Yeah I’m trying out 1.2.16, I’ll see if that works out. I saw something about updates though 1.6 being back ported to that or something, so figured I’d try it. Thanks!