<#5594 Updating annotations of pods belonging Ray ...
# flyte-github
c
#5594 Updating annotations of pods belonging Ray cluster in order to adopting Yunikorn Gang scheduling Pull request opened by 0yukali0 on <!date^1722001092^{date_short}|2024-07-26T13:38:12Z> ## Tracking issue Related to #5575 ## Why are the changes needed? Ability to classify pods to different gang scheduling group based on their role and worker group number. Currently, Pods under ray cluster share same gang scheduling task group via podTemplate. With same pod spec, pods share same gang scheduling group name when adopting yunikorn. ## What changes were proposed in this pull request? For enable gang scheduling with yunikorn, updating annotations pods belonging to ray cluster based on role and number of worker group. ## How was this patch tested? run make test_unit and make lint in flyteplugins. ### Setup process 1. adopting yaml including scheduler name and parameters
Copy code
tasks:
  task-plugins:
    enabled-plugins:
      - container
      - sidecar
      - K8S-ARRAY
      - ray
    default-for-task-types:
      - container: container
      - container_array: K8S-ARRAY
      - ray: ray
    
plugins:
  ray:
    batchScheduler:
      scheduler: yunikorn
      parameters: "placeholderTimeoutInSeconds=10 gangSchedulingStyle=Soft"
1. Set flyte dev env 2. Yunikorn installation embedAdmissionController: false enableSchedulerPlugin: false enableWebService: false
helm install yunikorn yunikorn/yunikorn -f values.yaml -n yunikorn --create-namespace
4. Set flyte plugin ray 5.Submit ray example
Copy code
import typing
from flytekit import ImageSpec, Resources, task, workflow

custom_image = ImageSpec(
        registry="localhost:30000",
    packages=["flytekitplugins-ray"],
    apt_packages=["wget"],
)

import ray
from flytekitplugins.ray import HeadNodeConfig, RayJobConfig, WorkerNodeConfig

@ray.remote
def f(x):
    return x * x

ray_config = RayJobConfig(
    head_node_config=HeadNodeConfig(ray_start_params={"log-color": "True"}),
    worker_node_config=[WorkerNodeConfig(group_name="ray-group", replicas=1), WorkerNodeConfig(group_name="ray-group2", replicas=1)],
    runtime_env={"pip": ["numpy", "pandas"]},  # or runtime_env="./requirements.txt"
    enable_autoscaling=False,
    shutdown_after_job_finishes=True,
    ttl_seconds_after_finished=3600,
)

@task(
    task_config=ray_config,
    requests=Resources(mem="2Gi", cpu="2"),
    container_image=custom_image,
)
def ray_task(n: int) -> typing.List[int]:
    futures = [f.remote(i) for i in range(n)]
    return ray.get(futures)

@workflow
def ray_workflow(n: int) -> typing.List[int]:
    return ray_task(n=n)

if __name__ == "__main__":
    print(ray_workflow(n=10))
### Screenshots [craete](https://private-user-images.githubusercontent.com/45888688/352532946-91df8233-a94b-48cc-a9ee-c22972ccd82f.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTM1NzUzOTcsIm5iZiI6MTc1MzU3NTA5NywicGF0aCI6Ii80NTg4ODY4OC8zNTI1MzI5NDYtOTFkZjgyMzMtYTk0Yi00OGNjLWE5ZWUtYzIyOTcyY2NkODJmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA3MjclMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwNzI3VDAwMTEzN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTk0Y2UyMDBiZmZiOTU2MjE1MWQxMDAzMWYxYzYzZjU4ZmYyMjYwMTkxNDU3YTQ1MGY3ZWI5ZTk3NDgxZTNhYTAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.McN0crxoctXtrhlitg4g2tbAmLz4pJ2UYe6cR8B4srI) [yunikornlog](https://private-user-images.githubusercontent.com/45888688/352551874-ef43dd56-82ae-493b-b420-405e1bb5a8e4.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTM1NzUzOTcsIm5iZiI6MTc1MzU3NTA5NywicGF0aCI6Ii80NTg4ODY4OC8zNTI1NTE4NzQtZWY0M2RkNTYtODJhZS00OTNiLWI0MjAtNDA1ZTFiYjVhOGU0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA3MjclMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwNzI3VDAwMTEzN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWE0NTM1ZWRjZWUwOTI3OTg3Y2M1MmY2NGIwNGEzZGNiYWNjMGY2YzhiYzU3ZmQwODdhZTMxNDNiNmViNDg5ODcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.tpnkVzoZNOIfMNLDsSsIdsxhlGCd7DpyHl1G4gxEGNA) [results](https://private-user-images.githubusercontent.com/45888688/352532990-966da92d-3ea0-49a4-94f5-317f21344094.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTM1NzUzOTcsIm5iZiI6MTc1MzU3NTA5NywicGF0aCI6Ii80NTg4ODY4OC8zNTI1MzI5OTAtOTY2ZGE5MmQtM2VhMC00OWE0LTk0ZjUtMzE3ZjIxMzQ0MDk0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA3MjclMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwNzI3VDAwMTEzN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTA5OTJiMDg5MmU4YmZhYzUzOTYwMTY3NTYxNGEwYTBmZWY5MWI1NWZmZmIyNmNjN2ZmOThhNTNiMjVmZjdmOGImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.H5JEk7VvmrPMgiqxZDrmAswtgxs_f1JbZIKyVDmLLGQ) chments/assets/d3d7d765-598f-4fcc-ae40-86bb8d684681) ## Check all the applicable boxes • I updated the documentation accordingly. • All new and existing tests passed. • All commits are signed-off. ## Related PRs ## Docs link Yunikorn gang scheduling doc flyteorg/flyte