Ketan (kumare3)
Kevin Su
06/09/2023, 7:56 PMKeshi Dai
06/12/2023, 9:48 PMKetan (kumare3)
Keshi Dai
06/13/2023, 12:38 AMray.init
) in RayFunctionTaskKetan (kumare3)
Keshi Dai
06/13/2023, 12:48 AMKevin Su
06/13/2023, 12:54 AMKeshi Dai
06/13/2023, 12:58 AMKevin Su
06/13/2023, 1:01 AMKeshi Dai
06/13/2023, 1:01 AMtherefore, actually, we run RayFunctionTask in the head node.Does that mean whatever defined under user’s @task decorator will be executed on the head node? In our case, Ray cluster and Flyte cluster are running on different GKE instances
ray.init(address=self._task_config.address)
is run in the flyte container right? IIUCaddress
parameter will be an IP from a pod in a different GKE instance. If that’s the case, it basically uses the Ray client to open a remote connection to the Ray clusteron flyte side, we basically job create a RayJobs CRD.In kuberay, a RayJob comes with a Ray python script, and usually is passed in via the configmap (see example here), I don’t think Flyte backend does that right?
Kevin Su
06/13/2023, 1:18 AMpyflyte-fast-execute --module wf --task ray_task
, and flytekit will download the code before running the ray task.Keshi Dai
06/13/2023, 1:21 AMTo run the task code, you have to build the image for ray cluster, and this image contain the code.in this way, the code will not be the one defined under @task decorator? we should package the ray python script into the image? sorry maybe I missed something here
Ketan (kumare3)
Kevin Su
06/13/2023, 1:32 AMKeshi Dai
06/13/2023, 1:36 AMKetan (kumare3)
Keshi Dai
06/13/2023, 1:55 AM