cuddly-jelly-27016
07/27/2025, 12:11 AMtasks:
task-plugins:
enabled-plugins:
- container
- sidecar
- K8S-ARRAY
- ray
default-for-task-types:
- container: container
- container_array: K8S-ARRAY
- ray: ray
plugins:
ray:
batchScheduler:
scheduler: yunikorn
parameters: "placeholderTimeoutInSeconds=10 gangSchedulingStyle=Soft"
1. Set flyte dev env
2. Yunikorn installation
embedAdmissionController: false
enableSchedulerPlugin: false
enableWebService: false
helm install yunikorn yunikorn/yunikorn -f values.yaml -n yunikorn --create-namespace
4. Set flyte plugin ray
5.Submit ray example
import typing
from flytekit import ImageSpec, Resources, task, workflow
custom_image = ImageSpec(
registry="localhost:30000",
packages=["flytekitplugins-ray"],
apt_packages=["wget"],
)
import ray
from flytekitplugins.ray import HeadNodeConfig, RayJobConfig, WorkerNodeConfig
@ray.remote
def f(x):
return x * x
ray_config = RayJobConfig(
head_node_config=HeadNodeConfig(ray_start_params={"log-color": "True"}),
worker_node_config=[WorkerNodeConfig(group_name="ray-group", replicas=1), WorkerNodeConfig(group_name="ray-group2", replicas=1)],
runtime_env={"pip": ["numpy", "pandas"]}, # or runtime_env="./requirements.txt"
enable_autoscaling=False,
shutdown_after_job_finishes=True,
ttl_seconds_after_finished=3600,
)
@task(
task_config=ray_config,
requests=Resources(mem="2Gi", cpu="2"),
container_image=custom_image,
)
def ray_task(n: int) -> typing.List[int]:
futures = [f.remote(i) for i in range(n)]
return ray.get(futures)
@workflow
def ray_workflow(n: int) -> typing.List[int]:
return ray_task(n=n)
if __name__ == "__main__":
print(ray_workflow(n=10))
### Screenshots
[craete](https://private-user-images.githubusercontent.com/45888688/352532946-91df8233-a94b-48cc-a9ee-c22972ccd82f.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTM1NzUzOTcsIm5iZiI6MTc1MzU3NTA5NywicGF0aCI6Ii80NTg4ODY4OC8zNTI1MzI5NDYtOTFkZjgyMzMtYTk0Yi00OGNjLWE5ZWUtYzIyOTcyY2NkODJmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA3MjclMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwNzI3VDAwMTEzN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTk0Y2UyMDBiZmZiOTU2MjE1MWQxMDAzMWYxYzYzZjU4ZmYyMjYwMTkxNDU3YTQ1MGY3ZWI5ZTk3NDgxZTNhYTAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.McN0crxoctXtrhlitg4g2tbAmLz4pJ2UYe6cR8B4srI)
[yunikornlog](https://private-user-images.githubusercontent.com/45888688/352551874-ef43dd56-82ae-493b-b420-405e1bb5a8e4.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTM1NzUzOTcsIm5iZiI6MTc1MzU3NTA5NywicGF0aCI6Ii80NTg4ODY4OC8zNTI1NTE4NzQtZWY0M2RkNTYtODJhZS00OTNiLWI0MjAtNDA1ZTFiYjVhOGU0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA3MjclMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwNzI3VDAwMTEzN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWE0NTM1ZWRjZWUwOTI3OTg3Y2M1MmY2NGIwNGEzZGNiYWNjMGY2YzhiYzU3ZmQwODdhZTMxNDNiNmViNDg5ODcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.tpnkVzoZNOIfMNLDsSsIdsxhlGCd7DpyHl1G4gxEGNA)
[results](https://private-user-images.githubusercontent.com/45888688/352532990-966da92d-3ea0-49a4-94f5-317f21344094.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTM1NzUzOTcsIm5iZiI6MTc1MzU3NTA5NywicGF0aCI6Ii80NTg4ODY4OC8zNTI1MzI5OTAtOTY2ZGE5MmQtM2VhMC00OWE0LTk0ZjUtMzE3ZjIxMzQ0MDk0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA3MjclMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwNzI3VDAwMTEzN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTA5OTJiMDg5MmU4YmZhYzUzOTYwMTY3NTYxNGEwYTBmZWY5MWI1NWZmZmIyNmNjN2ZmOThhNTNiMjVmZjdmOGImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.H5JEk7VvmrPMgiqxZDrmAswtgxs_f1JbZIKyVDmLLGQ)
chments/assets/d3d7d765-598f-4fcc-ae40-86bb8d684681)
## Check all the applicable boxes
• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.
## Related PRs
## Docs link
Yunikorn gang scheduling doc
flyteorg/flytecuddly-jelly-27016
07/27/2025, 12:11 AM