<#CP2HDHKE1|> <#CNMKCU6FR|> I am trying to allocat...
# flyte-support
s
#CP2HDHKE1 #CNMKCU6FR I am trying to allocate NVIDIA MIGS to a flyte job. The guide here https://docs.flyte.org/projects/cookbook/en/stable/auto/deployment/configure_use_gpus.html gives information only to allocate GPU's to flyte jobs. Migs however are like smaller chunks of an allocatable GPU which can be given to individual pods. I see that Flyte only has the default
Copy code
<http://nvidia.com/gpu|nvidia.com/gpu>
Is there any way to get Migs into this mix of things. @gifted-house-14547 I see that you were working on this 4 months back, did you get a solution to this issue. Please assist.
t
The gist link is broken, though. @gifted-house-14547, would you mind sharing the GitHub gist with us again, please?
g
@tall-lock-23197 sure, here are several example to allocate Nvidia MIG GPU (Remember that Multi Instance GPU can only be made with A100 or H100 (you can check here)
Copy code
"<http://nvidia.com/mig-3g.20gb|nvidia.com/mig-3g.20gb>":"1"

"<http://nvidia.com/mig-3g.7gb|nvidia.com/mig-3g.7gb>":"2"
floating number of gpu is impossible even if you are using MIG. Please feel free to ask anything about MIG
s
This gist link https://gist.github.com/SeungTaeKim-Qraft/35bfcf6104d0a735290b0ed8b6a1a1a0 is not working. Could you please assist with the working link @gifted-house-14547
g
you can check the sample codes above, @stocky-notebook-88311
s
Does the task decorator need to have the mig resource definition or the normal nvidia-GPU definition?
I see that I can mention container.resources=(mig-definition) in the pod_spec generator code but in the task decorator do i need to specify migs???
@gifted-house-14547, I dont see the sample codes in the link. your github gist link is not working
g
@stocky-notebook-88311 how about this example below
Copy code
resources = Resources(
        requests={"cpu": "1", "memory": "100Mi", "<http://nvidia.com/mig-1g.5gb|nvidia.com/mig-1g.5gb>":"1"}, 
        limits={"cpu": "1", "memory": "100Mi", "<http://nvidia.com/mig-1g.5gb|nvidia.com/mig-1g.5gb>":"1"},
        
    )
👍 1
you can assign requests & limits using Resource from flytekit module.
s
Thanks a lot @gifted-house-14547. Your help is greatly appreciated
🙏 1
f
This is great- would either of you be open to contributing to docs and also recommend if there is an improvement we can do to the container resources
@stocky-notebook-88311 / @gifted-house-14547
s
We are in the midst of implementing an ML platform with Flyte for workflows and inbuilt support for MIG's
f
Yes @stocky-notebook-88311 for example I think this resources is limiting. https://github.com/flyteorg/flytekit/blob/caf612d37e3767d6d0184d6d4e86c4a01f8adeab/flytekit/core/resources.py#L7
g
@freezing-airport-6809 I try to contribute this issue in this weekend
❤️ 2
💯 1
f
@gifted-house-14547 I also think that we should support resources that are string
👍 1
155 Views