James Chen
11/03/2023, 3:22 PMYicheng Lu
11/03/2023, 3:35 PMDoes this mean that Flyte is spinning up an ephemeral Ray cluster per-task instead of reusing an existing Ray cluster?Yes for the current plugin.
just using Ray where tasks with different dependencies in Ray are just provided a @ray.remote(runtime_env=foo), where foo provides a particular container image.This feature is not well supported in Ray, See https://ray-distributed.slack.com/archives/C02GFQ82JPM/p1693933328280079?thread_ts=1693857743.544139&cid=C02GFQ82JPM Using the flyte-ray plugin is essential mapping a flyte task to a ray job. See https://docs.ray.io/en/master/cluster/kubernetes/getting-started/rayjob-quick-start.html
James Chen
11/03/2023, 3:44 PMYicheng Lu
11/03/2023, 3:47 PMJames Chen
11/03/2023, 3:54 PMYicheng Lu
11/03/2023, 3:59 PMKevin Su
11/03/2023, 6:36 PM