freezing-airport-6809
glamorous-carpet-83516
06/09/2023, 7:56 PMrapid-autumn-97122
06/12/2023, 9:48 PMrapid-autumn-97122
06/12/2023, 9:49 PMfreezing-airport-6809
rapid-autumn-97122
06/13/2023, 12:38 AMray.init
) in RayFunctionTaskfreezing-airport-6809
freezing-airport-6809
rapid-autumn-97122
06/13/2023, 12:48 AMrapid-autumn-97122
06/13/2023, 12:49 AMglamorous-carpet-83516
06/13/2023, 12:54 AMrapid-autumn-97122
06/13/2023, 12:58 AMglamorous-carpet-83516
06/13/2023, 1:01 AMrapid-autumn-97122
06/13/2023, 1:01 AMtherefore, actually, we run RayFunctionTask in the head node.Does that mean whatever defined under user’s @task decorator will be executed on the head node? In our case, Ray cluster and Flyte cluster are running on different GKE instances
rapid-autumn-97122
06/13/2023, 1:02 AMray.init(address=self._task_config.address)
is run in the flyte container right? IIUCrapid-autumn-97122
06/13/2023, 1:04 AMaddress
parameter will be an IP from a pod in a different GKE instance. If that’s the case, it basically uses the Ray client to open a remote connection to the Ray clusterrapid-autumn-97122
06/13/2023, 1:05 AMon flyte side, we basically job create a RayJobs CRD.In kuberay, a RayJob comes with a Ray python script, and usually is passed in via the configmap (see example here), I don’t think Flyte backend does that right?
glamorous-carpet-83516
06/13/2023, 1:18 AMpyflyte-fast-execute --module wf --task ray_task
, and flytekit will download the code before running the ray task.rapid-autumn-97122
06/13/2023, 1:21 AMTo run the task code, you have to build the image for ray cluster, and this image contain the code.in this way, the code will not be the one defined under @task decorator? we should package the ray python script into the image? sorry maybe I missed something here
freezing-airport-6809
freezing-airport-6809
glamorous-carpet-83516
06/13/2023, 1:32 AMrapid-autumn-97122
06/13/2023, 1:36 AMfreezing-airport-6809
freezing-airport-6809
rapid-autumn-97122
06/13/2023, 1:55 AMrapid-autumn-97122
06/13/2023, 1:55 AM