And here is the default quota per project-domain. if you create too many ray workers, and the total cpu (number of workers * cpu per worker) exceeds the default quota, the task will always be in the queue status.

cluster_resources:
      customData:
      - production:
        - projectQuotaCpu:
            value: "5"
        - projectQuotaMemory:
            value: 4000Mi
      - staging:
        - projectQuotaCpu:
            value: "2"
        - projectQuotaMemory:
            value: 3000Mi
      - development:
        - projectQuotaCpu:
            value: "12"
        - projectQuotaMemory:
            value: 8000Mi