helpful-kilobyte-23008
03/28/2023, 2:16 PMflyteidl 1.3.12 depends on protobuf<5.0.0 and >=4.21.1
sagemaker 2.133.0 depends on protobuf<4.0 and >=3.1
Since 1.3 FlyteKit has version conflict with Sagemaker API and Tensorflow as well by deprecating protobuf 3 making it impossible to use those libraries togetherhelpful-kilobyte-23008
03/28/2023, 2:16 PMhelpful-kilobyte-23008
03/28/2023, 2:17 PMcool-lifeguard-49380
03/28/2023, 2:17 PMhelpful-kilobyte-23008
03/28/2023, 2:29 PMfreezing-airport-6809
freezing-airport-6809
freezing-airport-6809
helpful-kilobyte-23008
03/28/2023, 3:03 PMfreezing-airport-6809
jolly-whale-9142
03/28/2023, 3:10 PMhelpful-kilobyte-23008
03/28/2023, 3:11 PMfreezing-airport-6809
freezing-airport-6809
high-accountant-32689
03/28/2023, 6:44 PMWhy is it hard cutoff on protobuf 4? Can it be relaxed in newer version to include 3?As @cool-lifeguard-49380 pointed out, we're maintaining feature parity on the flytekit 1.2.x releases (including the flytekit plugins). We announced this change in flytekit 1.3.0 (check the release notes). I'll make sure to reinforce the message in all releases going forward. Also worth keeping in mind that tensorflow recently announced support for the next version of
protobuf
in their latest release (2.12.0).
For the sagemaker plugin specifically, thanks for bringing up the inconsistency. We're going to yank 1.3.x and 1.4.x versions of the sagemaker plugin from pypi as those are not really usable. This will clear up any confusion that people see when installing the plugin (since only versions 1.2.x and below will be available).
One final thing, as Ketan already mentioned, we're in the process of releasing flytekit backend plugins (you can follow the work in https://github.com/flyteorg/flyte/issues/3282). Sagemaker is a great candidate for such plugin.helpful-kilobyte-23008
03/30/2023, 9:48 AMhelpful-kilobyte-23008
03/30/2023, 9:49 AMfreezing-airport-6809
jolly-whale-9142
03/30/2023, 1:22 PMfreezing-airport-6809