https://flyte.org logo
#ray-integration
Title
# ray-integration
a

Anirudh Sridhar

09/06/2023, 12:50 PM
hey team, Ray and Flyte: Distributed Computing and Orchestration in this documentation it is written that
Flyte starts a Ray dashboard by default that provides cluster metrics and logs across many machines in a single pane as well as Ray memory utilization while debugging memory errors. The dashboard helps Ray users understand Ray clusters and libraries.
But i dont see ray dashboard i just see flyte console
m

Michael Tinsley

09/06/2023, 1:02 PM
You need to separately port forward to it, it should have a k8s service in whichever namespace its running
a

Anirudh Sridhar

09/06/2023, 1:05 PM
So does it not run in the demo cluster?
s

Samhita Alla

09/07/2023, 5:19 AM
cc @Kevin Su
k

Kevin Su

09/07/2023, 6:24 AM
yes, you can run ray task in the demo cluster, but you need to port-forward the ray service. kuberay won’t create an ingress for you, so you can only connect the dashboard through k8s service.
a

Anirudh Sridhar

09/07/2023, 10:09 AM
Hey @Kevin Su i am not able to find ray service i can just see all pods
h

Haytham Abuelfutuh

09/07/2023, 3:51 PM
While the task is running, can you run:
kubectl get svc -A
? and post the output here
a

Anirudh Sridhar

09/07/2023, 4:08 PM
Screenshot 2023-09-07 at 9.38.46 PM.png
h

Haytham Abuelfutuh

09/07/2023, 7:40 PM
This is odd... Can you check on two things for me plz: 1. I want to make sure the task actually ran using Ray operator, can you confirm that? maybe run
kubectl get rayjob -A
while the task is running 2. Can you confirm the version of Ray operator you are running?
a

Anirudh Sridhar

09/11/2023, 6:14 AM
Ray operator version:
kuberay/operator:v0.3.0
@Haytham Abuelfutuh
h

Haytham Abuelfutuh

09/11/2023, 9:45 PM
I see, @Anirudh Sridhar It doesn't look like Ray plugin is enabled... Have you followed the steps here: https://docs.flyte.org/en/latest/deployment/plugins/k8s/index.html? particularly the config:
Copy code
tasks:
  task-plugins:
    default-for-task-types:
      container: container
      container_array: k8s-array
      sidecar: sidecar
      ray: ray
    enabled-plugins:
    - container
    - k8s-array
    - sidecar
    - ray
a

Anirudh Sridhar

09/12/2023, 5:07 AM
ya it is enabled
3 Views