glamorous-pizza-44029
02/13/2025, 3:54 PMpyflyte register
command using the --image
tag, but I am not having any luck forcing the flytectl cluster to use the AMD64 architecture.
I started the demo specifically with this env variable flytectl demo start --env DOCKER_DEFAULT_PLATFORM=linux/amd64
(which theoretically should force docker to try to pull amd64 images?).
I am getting errors in my workflow that it can't find the container
[1/1] currentAttempt done. Last Error: USER::Grace period [3m0s] exceeded|containers with unready status: [afqwjjtbs6n4ltk4pd9l-n0-0]|Back-off pulling image "localhost:30000/my-image:latest"
Below is what I used to build/push/install the workflow:
docker build -t my-image:latest --platform=linux/amd64 .
docker tag detekt-challenge:latest localhost:30000/my-image:latest
docker push localhost:30000/my-image:latest
pyflyte register --project my-project --image localhost:30000/my-image:latest src
At this point I am not sure if this is an architecture issue or something else not working. I did verify I can pull the image back to my local machine (so the docker registry is running).
How can I test this locally with flytectl
?
Thanks 🙂average-finland-92144
02/13/2025, 5:05 PMDOCKER_DEFAULT_PLATFORM=linux/amd64I think this is for emulation. To build locally on M1 I have used ImageSpec and the
platform="linux/arm64"
argumentglamorous-pizza-44029
02/14/2025, 7:41 AMglamorous-pizza-44029
02/14/2025, 10:45 AMglamorous-pizza-44029
02/14/2025, 10:56 AMfierce-monitor-77717
04/03/2025, 1:25 PMglamorous-pizza-44029
04/03/2025, 1:28 PMbusy-holiday-5862
05/09/2025, 4:17 PME0509 16:11:11.973535 75 remote_image.go:180] "PullImage from image service failed" err="rpc error: code = NotFound desc = failed to pull and unpack image \"localhost:30000/say-hello-image:Y0RqVPnX2_221ovInaIIqw\": no match for platform in manifest: not found" image="localhost:30000/say-hello-image:Y0RqVPnX2_221ovInaIIqw"
But I can run the image (via Docker Desktop with Rosetta enabled), albeit with a warning:
$ docker run --rm localhost:30000/say-hello-image:Y0RqVPnX2_221ovInaIIqw echo 'hello'
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
hello
Slightly modified `hello_world.py`:
# Hello World
import flytekit as fl
image_spec = fl.ImageSpec(
# The name of the image. This image will be used by the `say_hello`` task.
name="say-hello-image",
# Lock file with dependencies to be installed in the image.
requirements="uv.lock",
# Image registry to to which this image will be pushed.
# Set the Environment variable FLYTE_IMAGE_REGISTRY to the URL of your registry.
# The image will be built on your local machine, so enure that your Docker is running.
# Ensure that pushed image is accessible to your Flyte cluster, so that it can pull the image
# when it spins up the task container.
registry="localhost:30000",
)
@fl.task(container_image=image_spec)
def say_hello(name: str) -> str:
return f"Hello, {name}!"
@fl.workflow
def hello_world_wf(name: str = "world") -> str:
greeting = say_hello(name=name)
return greeting
Adding platform="linux/arm64"
to the ImageSpec
works 🎉 (thank you for that), it might be useful to add that to the docs. It'd also be useful to point out the bundled image registry on localhost:30000
in the getting started guide as well, as that's great for local testing but not advertised in the guide.Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.
Powered by