cuddly-jelly-27016
08/25/2025, 12:10 AMdocker login
• general reqs:
• kustomize
• helm
• kubectl
• docker
• flyte envd reqs:
• pip install flytekitplugins-envd
• see below: create_envd_context.sh and run that
My env: (may or may not be necessary)
• /etc/docker/daemon.json
{
"default-runtime": "nvidia",
"runtimes": {
"nvidia": {
"args": [],
"path": "nvidia-container-runtime"
}
}
}
• docker context list
NAME DESCRIPTION DOCKER ENDPOINT ERROR
default * Current DOCKER_HOST based configuration unix:///var/run/docker.sock
• /etc/containerd/config.toml
version = 2
[plugins]
[plugins."io.containerd.grpc.v1.cri"]
[plugins."io.containerd.grpc.v1.cri".containerd]
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes]
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.nvidia]
privileged_without_host_devices = false
runtime_engine = ""
runtime_root = ""
runtime_type = "io.containerd.runc.v2"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.nvidia.options]
BinaryName = "/usr/bin/nvidia-container-runtime"
SystemdCgroup = true
• lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.3 LTS
Release: 22.04
Codename: jammy
• nvidia-smi
Wed Nov 1 03:52:14 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 545.23.06 Driver Version: 545.23.06 CUDA Version: 12.3 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 Tesla T4 On | 00000000:00:04.0 Off | 0 |
| N/A 33C P8 9W / 70W | 2MiB / 15360MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 1 Tesla T4 On | 00000000:00:05.0 Off | 0 |
| N/A 32C P8 9W / 70W | 2MiB / 15360MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| No running processes found |
+---------------------------------------------------------------------------------------+
# 1. Get branch
Download the branch, build the dockerfile, tag the image, and push it:
git clone <https://github.com/danpf/flyte>
git checkout danpf-sandbox-gpu
cd flyte/docker/sandbox-bundled
make build-gpu
docker tag flyte-sandbox-gpu:latest dancyrusbio/flyte-sandbox-gpu:latest
docker login
docker push dancyrusbio/flyte-sandbox-gpu:latest
# 2. Start the cluster
flytectl demo start --image dancyrusbio/flyte-sandbox-gpu:latest --disable-agent --force
# 3. See if you can use the gpu
$ kubectl describe node | grep -i gpu
<http://nvidia.com/gpu|nvidia.com/gpu>: 2
<http://nvidia.com/gpu|nvidia.com/gpu>: 2
<http://nvidia.com/gpu|nvidia.com/gpu> 0 0
# 4. run the final job:
create the runme.py script shown below, and then run
pyflyte run --remote runme.py check_if_gpu_available
## Testing scripts
# create_envd_context.sh
envd context create --name flyte-sandbox --builder tcp --builder-address localhost:30003 --use
quickly rebuild and push your docker image (change the name obviously)
# rebuild.sh
make build-gpu && docker tag flyte-sandbox-gpu dancyrusbio/flyte-sandbox-gpu && docker push dancyrusbio/flyte-sandbox-gpu
start a new flyte sandbox cluster
# start_new_flyte_cluster.sh
flytectl demo start --image dancyrusbio/flyte-sandbox-gpu:latest --disable-agent --force
This is the final flyte script to check if your gpu is working
# runme.py
from flytekit import ImageSpec, Resources, task
gpu = "1"
@task(
retries=2,
cache=True,
cache_version="1.0",
requests=Resources(gpu=gpu),
environment={"PYTHONPATH": "/root"},
container_image=ImageSpec(
cuda="11.8.0",
python_version="3.9.13",
packages=["flytekit", "torch"],
apt_packages=["git"],
registry="localhost:30000",
)
)
def check_if_gpu_available() -> bool:
import torch
return torch.cuda.is_available()
## Proof!
<https://private-user-images.githubusercontent.com/16297104/279567405-031e90ea-67c0-4b42-9b72-d6e60e8fe077.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTYwODA4OTgsIm5iZiI6MTc1NjA4MDU5OCwicGF0aCI6Ii8xNjI5NzEwNC8yNzk1Njc0MDUtMDMxZTkwZWEtNjdjMC00YjQyLTliNzItZDZlNjBlOGZlMDc3LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA4MjUlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwODI1VDAwMDk1OFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTdlODkwMTg3ODMzNjEwMTY3NjNiYWU3OTVhZWUwY2Y5NWJkOGJkZTg1YmJjZDk2MWRhOTc2MDMyMjA2OGIxMjEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.qlrihr-_2NSUrfecntPxipUPCPOcgiRIdM9m8pJ0KT4|[image](https://private-user-images.githubusercontent.com/16297104/279567405-031e90ea-67c0-4b42-9b72-d6e60e8fe077.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ…
flyteorg/flytecuddly-jelly-27016
08/25/2025, 12:10 AM