https://flyte.org logo
d

David Xia

09/19/2022, 6:42 AM
Hi @Ketan (kumare3), @Yee, @Kevin Su. Thanks for chatting with @Keshi Dai, Abdullah, and me on Sept 7 about the Ray plugin and our use case at Spotify. I finally got around to checking the use of passing custom Python objects between Flyte and its Ray plugin. Here's a minimal repro of the issue I saw with the latest flytekit pre-release. Lmk if I'm doing anything wrong or it's actually a bug. Thanks! 🙏
k

Ketan (kumare3)

09/19/2022, 1:28 PM
Do in the custom ray task example - you are not using the ray config right? https://github.com/davidxia/flyte-ray-custom-objects-issue-repro/blob/main/custom_objects_with_ray_task.py
k

Kevin Su

09/19/2022, 1:34 PM
I found that the dataclass was serialized to an empty literal for some reason when using ray. still investigate on it.
k

Ketan (kumare3)

09/19/2022, 1:56 PM
Ray might be patching dataclass?
k

Kevin Su

09/19/2022, 2:47 PM
I think so
d

David Xia

09/19/2022, 7:03 PM
@Ketan (kumare3)
you are not using the ray config right?
Thanks for catching. I wasn't. Updated to use it. Same errors though. I guess the
ray_config
didn't matter here because flyte was still creating local Ray cluster just not using
ray_config
to configure it.
hey, just curious if any updates on this 🙏
3 Views