<#CP2HDHKE1|ask-the-community> <#CNMKCU6FR|announc...
# ask-the-community
s
#ask-the-community #announcements I am trying to allocate NVIDIA MIGS to a flyte job. The guide here https://docs.flyte.org/projects/cookbook/en/stable/auto/deployment/configure_use_gpus.html gives information only to allocate GPU's to flyte jobs. Migs however are like smaller chunks of an allocatable GPU which can be given to individual pods. I see that Flyte only has the default
Copy code
<http://nvidia.com/gpu|nvidia.com/gpu>
Is there any way to get Migs into this mix of things. @SeungTaeKim I see that you were working on this 4 months back, did you get a solution to this issue. Please assist.
s
The gist link is broken, though. @SeungTaeKim, would you mind sharing the GitHub gist with us again, please?
s
@Samhita Alla sure, here are several example to allocate Nvidia MIG GPU (Remember that Multi Instance GPU can only be made with A100 or H100 (you can check here)
Copy code
"<http://nvidia.com/mig-3g.20gb|nvidia.com/mig-3g.20gb>":"1"

"<http://nvidia.com/mig-3g.7gb|nvidia.com/mig-3g.7gb>":"2"
floating number of gpu is impossible even if you are using MIG. Please feel free to ask anything about MIG
s
This gist link https://gist.github.com/SeungTaeKim-Qraft/35bfcf6104d0a735290b0ed8b6a1a1a0 is not working. Could you please assist with the working link @SeungTaeKim
s
you can check the sample codes above, @Sujith Samuel
s
Does the task decorator need to have the mig resource definition or the normal nvidia-GPU definition?
I see that I can mention container.resources=(mig-definition) in the pod_spec generator code but in the task decorator do i need to specify migs???
@SeungTaeKim, I dont see the sample codes in the link. your github gist link is not working
s
@Sujith Samuel how about this example below
Copy code
resources = Resources(
        requests={"cpu": "1", "memory": "100Mi", "<http://nvidia.com/mig-1g.5gb|nvidia.com/mig-1g.5gb>":"1"}, 
        limits={"cpu": "1", "memory": "100Mi", "<http://nvidia.com/mig-1g.5gb|nvidia.com/mig-1g.5gb>":"1"},
        
    )
you can assign requests & limits using Resource from flytekit module.
s
Thanks a lot @SeungTaeKim. Your help is greatly appreciated
k
This is great- would either of you be open to contributing to docs and also recommend if there is an improvement we can do to the container resources
@Sujith Samuel / @SeungTaeKim
s
We are in the midst of implementing an ML platform with Flyte for workflows and inbuilt support for MIG's
k
s
@Ketan (kumare3) I try to contribute this issue in this weekend
k
@SeungTaeKim I also think that we should support resources that are string
105 Views