Hi @Ketan (kumare3), @Yee, @Kevin Su. Thanks for chatting with @Keshi Dai, Abdullah, and me on Sept 7 about the Ray plugin and our use case at Spotify. I finally got around to checking the use of passing custom Python objects between Flyte and its Ray plugin. Here's a minimal repro of the issue I saw with the latest flytekit pre-release. Lmk if I'm doing anything wrong or it's actually a bug. Thanks! 🙏

Do in the custom ray task example - you are not using the ray config right? https://github.com/davidxia/flyte-ray-custom-objects-issue-repro/blob/main/custom_objects_with_ray_task.py

I found that the dataclass was serialized to an empty literal for some reason when using ray. still investigate on it.

Ray might be patching dataclass?

I think so

@Ketan (kumare3)

you are not using the ray config right?

Thanks for catching. I wasn't. Updated to use it. Same errors though. I guess the

ray_config

didn't matter here because flyte was still creating local Ray cluster just not using

ray_config

to configure it.