https://flyte.org logo
#flytekit
Title
# flytekit
m

Matheus Moreno

09/12/2022, 6:49 PM
Hey, everyone. Quick question: how viable would it be to bump Pyarrow to a greater version (such as 7.*)? We''re developing an internal ML library with a set of tools and I just bumped into a dependency problem: some Pyarrow functions we use were introduced at 7.0.0, but Flytekit requires version 6.0.1. According to the official release, 7.0.0 biggest change is the drop in support for Python 3.6. Since Flytekit's official support is >=3.7, this shouldn't be an issue, right?
y

Yee

09/12/2022, 7:22 PM
can you make the PR? but run requirements using python 3.7
m

Matheus Moreno

09/12/2022, 8:00 PM
ok, will do
A lot of dependencies changed 😅 but mostly in minor/patch levels
I am being unable to run
make requirements
. I'm getting this error:
Copy code
Could not find a version that matches flatbuffers>=2.0,~=1.12 (from tf2onnx==1.12.0->-r <http://doc-requirements.in|doc-requirements.in> (line 38))
Tried: 1.9, 1.9, 1.10, 1.10, 1.11, 1.11, 1.12, 1.12, 2.0, 2.0, 2.0.7, 2.0.7
There are incompatible versions in the resolved dependencies:
  flatbuffers>=2.0 (from tensorflow==2.10.0->-r <http://doc-requirements.in|doc-requirements.in> (line 39))
  flatbuffers~=1.12 (from tf2onnx==1.12.0->-r <http://doc-requirements.in|doc-requirements.in> (line 38))
Even without changing anything in the code (i.e. not bumping Pyarrow), this error persists
y

Yee

09/12/2022, 9:38 PM
can you just push changes to the other files for now?
this is only for building docs.
i saw this last week when testing.
m

Matheus Moreno

09/13/2022, 2:16 PM
oh ok
tried bumping to 9.0.0, then 8.0.0 then 7.0.0. all versions are failing with
ImportError: cannot import name '_message' from 'google.protobuf.pyext' (/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/site-packages/google/protobuf/pyext/__init__.py)
, which doesn't necessarily seem like an error from Pyarrow itself. Maybe when I'm running
make requirements
some other library is being updated and affecting the compatibility?
26 Views