I know the ray plugin typically spins up an an ephemeral ray cluster to run a ray job (for isolation purposes). Does the flyte-ray plugin support running ray jobs on an already-running static ray cluster? If so, how would one produce that functionality? Thank you!

you could use ray.init in the python task to submit the ray task to the existing cluster. like

@task
def t1():
    ray.init(<ip_address>)
    ray.remote(...)

I notice that you're calling Ray directly inside the task, is there a reason you're not using the actual flytekitplugins-ray plugin (https://pypi.org/project/flytekitplugins-ray/)?

The flytekitplugins-ray plugin is used to create ray clusters in flyte.

Oh, I thought I could also connect to an already-running cluster via an

address

. In the latest plugin source code (

flytekitplugins/ray/task.py

) I see this `dataclass`:

@dataclass
class RayJobConfig:
    worker_node_config: typing.List[WorkerNodeConfig]
    head_node_config: typing.Optional[HeadNodeConfig] = None
    runtime_env: typing.Optional[dict] = None
    address: typing.Optional[str] = None

I would expect this address to work to send a job to an existing Ray Cluster deployed on Kubernetes AWS:

address="<ray://ray-cluster-kuberay-head-svc.ray-cluster-namespace.svc.cluster.local:10001>"

It seems like regardless of weather the

address

field is present or not, the plugin spins up a new ephemeral Ray cluster. I did get it working via your example without the plugin @Kevin Su but I was just curious if there was a way to do this via the plugin. Thanks for your help!