Hello ^_^ Before trying this out myself, has there been any attempts/thoughts on enabling nvidia-acceleration for the single container demo cluster?

No, there has been no attempts. If you happen to do it please share 🤯

Ok, got it working finally... based the sandbox on

nvidia/cuda:11.3.0-runtime-ubuntu20.04

instead of

rancher/k3s:v1.24.4-k3s1

and installed k3s and crictl during docker build... haven't worked with docker buildx before, so not sure how to beautify it ^_^ will see if I can clean it up a little bit further

Cc @jeev / @Yee

Neat. We could also possibly consider a “multi-node” setup where one of the nodes has GPU drivers and can be fired up in an opt-in way. GPU workloads can then be configured to schedule on this node via the usual affinity.

How would you prefer to have it delivered? As a PR or just as files that you can fit into the build system according to your wishes? If PR, should I add it as a build target for the Makefile in sandbox-bundled or make a new sandbox directory you think?

i’m thinking probably a build target in sandbox-bundled. if you can open a draft PR, we can discuss further. Perhaps we can reorganize the current default stage to be based on some version of ubuntu that would make layering GPU drivers+cuda on easier too.

Will do!

cc @jeev / @Yee will need your expertise

To run the image you need to have an nvidia-enabled docker... I did this by installing (in ubuntu 20.04)

nvidia-docker2

and use

/etc/docker/daemon.json

like so:

{
  "runtimes": {
    "nvidia": {
      "path": "nvidia-container-runtime",
      "runtimeArgs": []
    }
  }
}

and the restart docker

we will need to run on a cloud instance with gpu 🙂

are y’all running this on developer machines @Björn?

@jeev During development I have been running this on a workstation at home... Hopefully our data scientists can run this either or their gpu enabled workstations or on a vertex ai workbench in GCP with attached T4 GPU

awesome thanks. will take a look tomorrow 🙂