Question about the Ray plugin. In the docs, it mentions that the Ray plugin use Ray's Job Submission to run the ray task. However, looking at the code, I don't see the

JobSubmissionClient

being used anywhere. Am I missing something?

Kuberay will do that

Flyte creates RayJob custom resource

@Kevin Su @Ketan (kumare3) Does KubeRay’s RayJob rely on Ray’s job submission API? IIUC, it seems not because Flyte’s Ray plugin still uses Ray client ray.init to interact with Ray cluster created via the plugin

who brings up the cluster, in kuberays case it brings up the cluster first and then submits the job

Yeah, but I meant in your ray flyte plugin backend, it only brings up a ray cluster, the actual job task submission is done via the Ray client (

ray.init

) in RayFunctionTask

Does that not need access to the python code

It’s working for some use cases. The downsides are 1) it forces flyte/ray env consistency 2) Ray client is recommended for the interactive dev, and it will eventually go away.

if you use ray task in flytekit, we will create a rayJob CRD, kuberay will create a cluster first, and then use ray job submission api to submit the ray job to the head node. therefore, actually, we run RayFunctionTask in the head node. do you want us to launch a separate pod that uses ray client to submit the job to the ray cluster?

To be clear, we are not asking for anything. We just would like to understand how this job gets executed on the flyte backend, so we can design things accordingly.

I see, on flyte side, we basically job create a RayJobs CRD.

therefore, actually, we run RayFunctionTask in the head node.

Does that mean whatever defined under user’s @task decorator will be executed on the head node? In our case, Ray cluster and Flyte cluster are running on different GKE instances

no, we don’t add the code to the config map. To run the task code, you have to build the image for ray cluster, and this image contain the code. another way is to use fast-register in flytekit. the entrypoint of ray job will become something like

pyflyte-fast-execute --module wf --task ray_task

, and flytekit will download the code before running the ray task.

To run the task code, you have to build the image for ray cluster, and this image contain the code.

in this way, the code will not be the one defined under @task decorator? we should package the ray python script into the image? sorry maybe I missed something here

I think you folks are talking over each other

you still write the ray code in the task decorator. it just likes running the regular python task. you have to build the image for you task / workflow. In the flytekit ray task, the image will used in the rayJob CRD.

Gotcha thanks!

So to quickly recap, Flyte will not run a pod, we simply create a kuberay crd, it creates the driver and worker with flytekit entrypoint. So it is not client mode, cluster mode. Code is either built into the container or sent using fast registration

👍 sounds great!