Hey Flyte Gang 🙂 I am a huge fan of the new ImageSpec Feature and wanted to play with it a little. I can build Images but pushing it to an AWS ECR repo is not working for me atm.

pandas_image_spec = ImageSpec(
    base_image="<http://ghcr.io/flyteorg/flytekit:py3.8-1.6.2|ghcr.io/flyteorg/flytekit:py3.8-1.6.2>",
    packages=["pandas", "numpy"],
    python_version="3.9",
    apt_packages=["git"],
    env={"Debug": "True"},
    registry="<http://account_id.dkr.ecr.eu-central-1.amazonaws.com/jansdataplane-dev-playground-base|account_id.dkr.ecr.eu-central-1.amazonaws.com/jansdataplane-dev-playground-base>",
)

pyflyte build img_spec_test.py wf

is giving me an error someting like this:

Post "<https://account_id.dkr.ecr.eu-central-1.amazonaws.com/v2/jansdataplane-dev-playground-base/flytekit/blobs/uploads/>": EOF

Is ECR supported on this one?

Don't know about the Post API stuff, but from what I see from the docs and code around it, it seems to work with the underlying docker engine. Do you know whether that one has access and permissions for the registry you specified?

Yes locally i tested my access and manual docker push worked after logging in via

aws ecr get-login-password --region eu-central-1 | docker login --username AWS --password-stdin <http://account_id.dkr.ecr.eu-central-1.amazonaws.com|account_id.dkr.ecr.eu-central-1.amazonaws.com>

Is there more stack trace to the error?

All i have is this:

Failed with Unknown Exception <class 'Exception'> Reason: failed to run command envd build --path /var/folders/k6/br3z_w6d797cf9rwtztxzzb80000gn/T/flyte-wo8bfqlu/sandbox/local_flytekit/18cc84e1681acb1ea991921f5331e3e0  --platform linux/amd64 --output type=image,name=<http://account_id.dkr.ecr.eu-central-1.amazonaws.com/jansdataplane-dev-playground-base/flytekit:ezWA_dNcNWhWrZxn7xhxMg..,push=true|account_id.dkr.ecr.eu-central-1.amazonaws.com/jansdataplane-dev-playground-base/flytekit:ezWA_dNcNWhWrZxn7xhxMg..,push=true> with error b'error: failed to do request: Post "<https://account_id.dkr.ecr.eu-central-1.amazonaws.com/v2/jansdataplane-dev-playground-base/flytekit/blobs/uploads/>": EOF\n'

failed to run command envd build --path /var/folders/k6/br3z_w6d797cf9rwtztxzzb80000gn/T/flyte-wo8bfqlu/sandbox/local_flytekit/18cc84e1681acb1ea991921f5331e3e0  --platform linux/amd64 --output type=image,name=<http://account_id.dkr.ecr.eu-central-1.amazonaws.com/jansdataplane-dev-playground-base/flytekit:ezWA_dNcNWhWrZxn7xhxMg..,push=true|account_id.dkr.ecr.eu-central-1.amazonaws.com/jansdataplane-dev-playground-base/flytekit:ezWA_dNcNWhWrZxn7xhxMg..,push=true> with error b'error: failed to do request: Post "<https://account_id.dkr.ecr.eu-central-1.amazonaws.com/v2/jansdataplane-dev-playground-base/flytekit/blobs/uploads/>": EOF\n'

Some pointers I stumbled upon: ImageSpec uses by default envd as a builder The expection came from here, where the command was build here why/how the API request was build I don't know. The EOF in the context of docker usually comes on upload when the repository doesn't exists, which it doesn't with the malformed url

cc @Keming Yang @Ce Gao

I think you need to provide credentials. It works locally because you used docker login. ImageSpec is running a buildkit daemon inside a Docker container, which doesn't have access to your credentials. @Keming Yang would using moby worker fix this?

Thanks for clarifying! Would be nice to know how to make it work, since pushing to a private repository is a common thing to do, no?

buildkit container can also get the credentials stored in

$HOME/.docker/config.json

I also can see entries for the given ecr in .docker/config.json

{
	"auths": {
		"<http://account_id.dkr.ecr.eu-central-1.amazonaws.com|account_id.dkr.ecr.eu-central-1.amazonaws.com>": {},,
		"<http://azuredevplaygroundkaniko.azurecr.io|azuredevplaygroundkaniko.azurecr.io>": {},
		"<https://index.docker.io/v1/>": {}
	},
	"credsStore": "desktop"
}

envd build did not came up with additional errors. Its not super urgent for me atm but would be nice if this kind of use case would be documented in the future

is it related to https://stackoverflow.com/questions/45342682/how-to-fix-eof-error-during-docker-push-command?

Alright i made it work 🤩 A closer look at the URL should have done it in the first place. If you use the following ImageSpec:

pandas_image_spec = ImageSpec(
    base_image="ghcr.io/flyteorg/flytekit:py3.8-1.6.2",
    packages=["pandas", "numpy"],
    python_version="3.9",
    apt_packages=["git"],
    env={"Debug": "True"},
    registry="account_id.dkr.ecr.eu-central-1.amazonaws.com",
)

pyflyte build expects a repository named

flytekit

in ecr:

<http://account_id.dkr.ecr.eu-central-1.amazonaws.com/flytekit|account_id.dkr.ecr.eu-central-1.amazonaws.com/flytekit>

and pushes it there. Would be cool to choose the repository name myself and not use

flytekit

all the time

you can specify the name in the image spec. The default is flytekit

ImageSpec(name=<repository name>)