Andrew Korzhuev
03/28/2023, 2:16 PMflyteidl 1.3.12 depends on protobuf<5.0.0 and >=4.21.1
sagemaker 2.133.0 depends on protobuf<4.0 and >=3.1
Since 1.3 FlyteKit has version conflict with Sagemaker API and Tensorflow as well by deprecating protobuf 3 making it impossible to use those libraries togetherFabio Grätz
03/28/2023, 2:17 PMAndrew Korzhuev
03/28/2023, 2:29 PMKetan (kumare3)
Andrew Korzhuev
03/28/2023, 3:03 PMKetan (kumare3)
Stephen
03/28/2023, 3:10 PMAndrew Korzhuev
03/28/2023, 3:11 PMKetan (kumare3)
Eduardo Apolinario (eapolinario)
03/28/2023, 6:44 PMWhy is it hard cutoff on protobuf 4? Can it be relaxed in newer version to include 3?As @Fabio Grätz pointed out, we're maintaining feature parity on the flytekit 1.2.x releases (including the flytekit plugins). We announced this change in flytekit 1.3.0 (check the release notes). I'll make sure to reinforce the message in all releases going forward. Also worth keeping in mind that tensorflow recently announced support for the next version of
protobuf
in their latest release (2.12.0).
For the sagemaker plugin specifically, thanks for bringing up the inconsistency. We're going to yank 1.3.x and 1.4.x versions of the sagemaker plugin from pypi as those are not really usable. This will clear up any confusion that people see when installing the plugin (since only versions 1.2.x and below will be available).
One final thing, as Ketan already mentioned, we're in the process of releasing flytekit backend plugins (you can follow the work in https://github.com/flyteorg/flyte/issues/3282). Sagemaker is a great candidate for such plugin.Andrew Korzhuev
03/30/2023, 9:48 AMKetan (kumare3)
Stephen
03/30/2023, 1:22 PMKetan (kumare3)