Hi, I am using the master version of kuberay-opera...
# ray-integration
p
Hi, I am using the master version of kuberay-operator. Commands used to deploy kuberay-operator are:
Copy code
git clone <https://github.com/ray-project/kuberay.git>
cd kuberay
kubectl create -k ray-operator/config/default
When I submit the Ray workflow, rayjob, head and worker pods are getting created and also is up and running. But the Workflow is not getting submitted in the cluster and the job is in queue in console for more than 1 hour. Script:
Copy code
import typing
import ray
import time
from flytekit import Resources, task, workflow
from flytekitplugins.ray import HeadNodeConfig, RayJobConfig, WorkerNodeConfig


@ray.remote
def square(x):
    return x * x

ray_config = RayJobConfig(
      head_node_config=HeadNodeConfig(ray_start_params={"log-color": "True"}),
      worker_node_config=[WorkerNodeConfig(group_name="ray-group", replicas=2)],
      runtime_env={"pip": ["numpy", "pandas"]},
)


@task(task_config=ray_config, requests=Resources(mem="2000Mi", cpu="1"), limits=Resources(mem="3000Mi", cpu="2"))
def ray_task(n: int) -> typing.List[int]:
    futures = [square.remote(i) for i in range(n)]
    return ray.get(futures)


@workflow
def ray_workflow(n: int) -> typing.List[int]:
    return ray_task(n=n)

if __name__ == "__main__":
    print(ray_workflow(n=10))
image.png
m
Hi, I am also facing the same issue. Anyone please tell us the solution
d
cc @Kevin Su
k
Hey @Padma Priya M are you using sandbox?
Could you try kuberay0.5.2 instead
p
I am using EKS cluster
sure will try with 0.5.2
k
Thanks
p
Copy code
export KUBERAY_VERSION=v0.5.2
kubectl create -k "<http://github.com/ray-project/kuberay/manifests/cluster-scope-resources?ref=${KUBERAY_VERSION}&timeout=90s|github.com/ray-project/kuberay/manifests/cluster-scope-resources?ref=${KUBERAY_VERSION}&timeout=90s>"
kubectl apply -k "<http://github.com/ray-project/kuberay/manifests/base?ref=${KUBERAY_VERSION}&timeout=90s|github.com/ray-project/kuberay/manifests/base?ref=${KUBERAY_VERSION}&timeout=90s>"
can you confirm on the installation steps
k
Yes, correct
we also run into same issue, investigating, will get back to you
p
sure thanks. v0.5.2 is working fine for me. v0.5.2 is stable version only right?
k
yes, it is.
cc @manikandan sankar could you try 0.5.2