Thread
#announcements
    s

    SeungTaeKim

    3 months ago
    Hi, I am trying to assign GPU (Nvidia MIG - Multi-Instance GPU) on flyte. However, I cannot see how I assign it on each tasks in flyte (there were no mentions in flyte docs). Does anyone have a solution to it?
    Samhita Alla

    Samhita Alla

    3 months ago
    Hi, @SeungTaeKim! You can assign GPU to a task as follows:
    @task(requests=Resources(gpu=1, mem="3Gi", storage="1Gi"))
    def pytorch_mnist_task(hp: Hyperparameters) -> TrainingOutputs:
       ...
    s

    SeungTaeKim

    3 months ago
    @Samhita Alla Ah, the point I want to ask is flyte recognize "nvidia MIG (splitted instances into one gpu)" when I assign the way you mentioned. the solution you said recieves integer parameter, not gpu instance name. I want to try to use MIG on flyte.
    Samhita Alla

    Samhita Alla

    3 months ago
    You can configure it here by following this guide.
    @Yuvraj / @katrina, does the aforementioned way of configuring GPU work for MIG?
    s

    SeungTaeKim

    3 months ago
    @Samhita Alla I have also read
    values.yaml
    file & guide docs, but these does not provide specific answers of MIG support. I think I have waited answers from those who you tagged
    y

    Yuvraj

    3 months ago
    @SeungTaeKim As per nvidiaโ€™s documentation, You will get different GPU in MIG with different name Like this, You can add them in config as mention here
    kubectl describe node
    ...
    Capacity:
    <http://nvidia.com/mig-1g.5gb|nvidia.com/mig-1g.5gb>:   1
    <http://nvidia.com/mig-2g.10gb|nvidia.com/mig-2g.10gb>:  1
    <http://nvidia.com/mig-3g.20gb|nvidia.com/mig-3g.20gb>:  1
    This is the global config for all task, Flyte will add taints and tolerations to all the task if they require GPU. Currently only Pod Task has option to configure taints and tolerations, Flyte core task doesnโ€™t have option to configure the taints and tolerations, it will always use the default GPU. cc: @Dan Rammer (hamersaw) @Haytham Abuelfutuh Please correct me if i am wrong.
    s

    SeungTaeKim

    3 months ago
    @Samhita Alla I think I got a solution to assign MIG on flyte. flyte provides Integrations with k8s plugin, which can define pod spec in the flyte codes. https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/kubernetes/pod/pod.html
    Samhita Alla

    Samhita Alla

    3 months ago
    Nice! Is that working? If so, could you share with us the pod spec you used?
    s

    SeungTaeKim

    3 months ago
    @Samhita Alla sure I will share the result soon! I think flyte docs should mention about MIG strategy if I succeeded.
    Samhita Alla

    Samhita Alla

    3 months ago
    Oh yes! Thank you.
    Ketan (kumare3)

    Ketan (kumare3)

    3 months ago
    Yes that would be awesome
    Also we could change the default Container plugin to use this strategy in a simple way
    Please share