Please could you provide some more information about adding packages and using custom images with ImageSpec. Specifically, I have packages internally here at work that I want to install from file. I started by editing the Dockerfile in the flyte repo and adding lines to copy and pip install the files here.

COPY my-packages/ /root/my-packages/
...
&& uv pip install --no-cache-dir --system /root/my-packages/my-package/ \
... # rest of the file

and then uploaded this to ECR. Then in imagespec I tried to use this as the base image

image_spec = ImageSpec(
    base_image="<aws-account-id>.<http://dkr.ecr.us-east-1.amazonaws.com/<ecr-repo>:py3.11-v1.15.0|dkr.ecr.us-east-1.amazonaws.com/<ecr-repo>:py3.11-v1.15.0>",
    packages=[...],
    python_version="3.11",
    apt_packages=["git"],
    env={"Debug": "True"},
    registry="<aws-account-id>.<http://dkr.ecr.us-east-1.amazonaws.com/flyte|dkr.ecr.us-east-1.amazonaws.com/flyte>",
)

but when I try and import the package from inside a

task

I get

ModuleNotFoundError: No module named 'my_package'

. I can see from the path error that the python used in the task is micromamba, which is not the same one as the flyte base image. And when I build an image with pyflyte, there seems to be layering of the images

<http://ghcr.io/astral-sh/uv:0.5.1|ghcr.io/astral-sh/uv:0.5.1>

, my image and

<http://docker.io/mambaorg/micromamba:2.0.3-debian12-slim|docker.io/mambaorg/micromamba:2.0.3-debian12-slim>

. It would be good to know what each of these images are doing. Or equally, what is the point of the

base_image

? Why is

flytekit

installed if a different flytekit can be installed in the micromamba layer? Note: although I am talking about a custom package, if I pip install any package (e.g. tested with numpyro), I cannot import this from within a task. Obviously, I can use the

package=

arg to install this instead, so it isn't as much of an issue. The main issue is that I need to install these packages into an image from file and so that I can access them in a task. Happy to test any commands you send. Cheers

hey @gorgeous-caravan-46442 if I understand your use case correctly, I think you may find this new feature useful: https://github.com/flyteorg/flytekit/pull/3231

I think so. I'm not sure. I think I need a little more info on the standard image builder and what this "noop" builder does? In particular, I'd like to understand how these three layered images (

<http://ghcr.io/astral-sh/uv:0.5.1|ghcr.io/astral-sh/uv:0.5.1>

, my image and

<http://docker.io/mambaorg/micromamba:2.0.3-debian12-slim|docker.io/mambaorg/micromamba:2.0.3-debian12-slim>

) are used