https://flyte.org logo
#ask-the-community
Title
# ask-the-community
s

Sujith Samuel

10/27/2022, 6:50 AM
#ask-the-community #announcements I am trying to allocate NVIDIA MIGS to a flyte job. The guide here https://docs.flyte.org/projects/cookbook/en/stable/auto/deployment/configure_use_gpus.html gives information only to allocate GPU's to flyte jobs. Migs however are like smaller chunks of an allocatable GPU which can be given to individual pods. I see that Flyte only has the default
Copy code
<http://nvidia.com/gpu|nvidia.com/gpu>
Is there any way to get Migs into this mix of things. @SeungTaeKim I see that you were working on this 4 months back, did you get a solution to this issue. Please assist.
s

Samhita Alla

10/27/2022, 7:09 AM
The gist link is broken, though. @SeungTaeKim, would you mind sharing the GitHub gist with us again, please?
s

SeungTaeKim

10/27/2022, 7:22 AM
@Samhita Alla sure, here are several example to allocate Nvidia MIG GPU (Remember that Multi Instance GPU can only be made with A100 or H100 (you can check here)
Copy code
"<http://nvidia.com/mig-3g.20gb|nvidia.com/mig-3g.20gb>":"1"

"<http://nvidia.com/mig-3g.7gb|nvidia.com/mig-3g.7gb>":"2"
floating number of gpu is impossible even if you are using MIG. Please feel free to ask anything about MIG
s

Sujith Samuel

10/27/2022, 7:24 AM
This gist link https://gist.github.com/SeungTaeKim-Qraft/35bfcf6104d0a735290b0ed8b6a1a1a0 is not working. Could you please assist with the working link @SeungTaeKim
s

SeungTaeKim

10/27/2022, 7:25 AM
you can check the sample codes above, @Sujith Samuel
s

Sujith Samuel

10/27/2022, 7:28 AM
Does the task decorator need to have the mig resource definition or the normal nvidia-GPU definition?
I see that I can mention container.resources=(mig-definition) in the pod_spec generator code but in the task decorator do i need to specify migs???
@SeungTaeKim, I dont see the sample codes in the link. your github gist link is not working
s

SeungTaeKim

10/27/2022, 7:31 AM
@Sujith Samuel how about this example below
Copy code
resources = Resources(
        requests={"cpu": "1", "memory": "100Mi", "<http://nvidia.com/mig-1g.5gb|nvidia.com/mig-1g.5gb>":"1"}, 
        limits={"cpu": "1", "memory": "100Mi", "<http://nvidia.com/mig-1g.5gb|nvidia.com/mig-1g.5gb>":"1"},
        
    )
you can assign requests & limits using Resource from flytekit module.
s

Sujith Samuel

10/27/2022, 7:32 AM
Thanks a lot @SeungTaeKim. Your help is greatly appreciated
k

Ketan (kumare3)

10/27/2022, 1:53 PM
This is great- would either of you be open to contributing to docs and also recommend if there is an improvement we can do to the container resources
@Sujith Samuel / @SeungTaeKim
s

Sujith Samuel

10/27/2022, 1:55 PM
We are in the midst of implementing an ML platform with Flyte for workflows and inbuilt support for MIG's
k

Ketan (kumare3)

10/27/2022, 1:59 PM
s

SeungTaeKim

10/27/2022, 8:55 PM
@Ketan (kumare3) I try to contribute this issue in this weekend
k

Ketan (kumare3)

10/28/2022, 12:30 AM
@SeungTaeKim I also think that we should support resources that are string
2 Views