https://flyte.org logo
#announcements
Title
# announcements
s

SeungTaeKim

06/07/2022, 4:22 AM
Hi, I am trying to assign GPU (Nvidia MIG - Multi-Instance GPU) on flyte. However, I cannot see how I assign it on each tasks in flyte (there were no mentions in flyte docs). Does anyone have a solution to it?
s

Samhita Alla

06/07/2022, 4:32 AM
Hi, @SeungTaeKim! You can assign GPU to a task as follows:
Copy code
@task(requests=Resources(gpu=1, mem="3Gi", storage="1Gi"))
def pytorch_mnist_task(hp: Hyperparameters) -> TrainingOutputs:
   ...
s

SeungTaeKim

06/07/2022, 4:45 AM
@Samhita Alla Ah, the point I want to ask is flyte recognize "nvidia MIG (splitted instances into one gpu)" when I assign the way you mentioned. the solution you said recieves integer parameter, not gpu instance name. I want to try to use MIG on flyte.
s

Samhita Alla

06/07/2022, 4:50 AM
You can configure it here by following this guide.
๐Ÿค” 1
@Yuvraj / @katrina, does the aforementioned way of configuring GPU work for MIG?
s

SeungTaeKim

06/07/2022, 5:17 AM
@Samhita Alla I have also read
values.yaml
file & guide docs, but these does not provide specific answers of MIG support. I think I have waited answers from those who you tagged
y

Yuvraj

06/07/2022, 6:52 AM
@SeungTaeKim As per nvidiaโ€™s documentation, You will get different GPU in MIG with different name Like this, You can add them in config as mention here
Copy code
kubectl describe node
...
Capacity:
<http://nvidia.com/mig-1g.5gb|nvidia.com/mig-1g.5gb>:   1
<http://nvidia.com/mig-2g.10gb|nvidia.com/mig-2g.10gb>:  1
<http://nvidia.com/mig-3g.20gb|nvidia.com/mig-3g.20gb>:  1
This is the global config for all task, Flyte will add taints and tolerations to all the task if they require GPU. Currently only Pod Task has option to configure taints and tolerations, Flyte core task doesnโ€™t have option to configure the taints and tolerations, it will always use the default GPU. cc: @Dan Rammer (hamersaw) @Haytham Abuelfutuh Please correct me if i am wrong.
๐Ÿ‘ 1
s

SeungTaeKim

06/07/2022, 8:44 AM
@Samhita Alla I think I got a solution to assign MIG on flyte. flyte provides Integrations with k8s plugin, which can define pod spec in the flyte codes. https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/kubernetes/pod/pod.html
s

Samhita Alla

06/07/2022, 8:50 AM
Nice! Is that working? If so, could you share with us the pod spec you used?
๐Ÿ‘ 1
s

SeungTaeKim

06/07/2022, 8:52 AM
@Samhita Alla sure I will share the result soon! I think flyte docs should mention about MIG strategy if I succeeded.
s

Samhita Alla

06/07/2022, 8:58 AM
Oh yes! Thank you.
๐Ÿ‘ 1
k

Ketan (kumare3)

06/07/2022, 4:04 PM
Yes that would be awesome
๐Ÿ™ 1
Also we could change the default Container plugin to use this strategy in a simple way
๐Ÿ™ 1
๐Ÿ‘ 1
Please share
๐Ÿ‘ 1
40 Views