Hello I ve been trying to avoid using `pyflyte register` and

Question

Hello I ve been trying to avoid using `pyflyte register` and instead using `pyflyte package` with `flytectl register` in order to have more control over what I package Currently my directory structure is the following ```root src workflows py flyte tasks py``` Inside workflows py i do `from src flyte tasks import task1` To register everything I m running ```pyflyte k src package f p lt PROJECT NAME gt d lt DEV gt image mycustomimage test flytectl register files p lt PROJECT NAME gt d lt DEV gt archive version lt VERSION gt destinationDirectory root flyte package tgz``` Inside `mycustomimage` I have set PYTHONPATH= root The problem is that once the workflow is registered and I run it in the Flyte UI I always get the following error ``` 1 1 currentAttempt done Last Error USER j load task loader args=resolver args opt venv lib python3 9 site packages flytekit core utils py 295 in wrapper 295 return func args kwargs opt venv lib python3 9 site packages flytekit core python auto container py 235 in load task 235 task module = importlib import module name=task module typ usr local lib python3 9 importlib init py 127 in import module 127 return bootstrap gcd import name level package level in gcd import 1030 in find and load 1007 in find and load unlocked 972 in call with frames removed 228 in gcd import 1030 in find and load 1007 in find and load unlocked 984 ModuleNotFoundError No module named src ``` Am I doing something wrong

Jan Fiedler · Answer

Can you share your full Dockerfile

Nuno Martins · Answer

Here it is ```FROM python 3 9 slim buster ARG tag ENV FLYTE INTERNAL IMAGE $tag Venv variables ENV VENV opt venv ENV LANG C UTF 8 ENV LC ALL C UTF 8 ENV PYTHONPATH root ENV PATH= $ VENV bin $PATH Path to the Flyte Client Secret file ENV FLYTE CLIENT SECRET FILE= flyte configs flyte secret Update apt packages RUN apt get update y amp amp apt get upgrade y amp amp apt get install y curl git gcc RUN rm rf var lib apt lists Install pip dependencies COPY requirements txt RUN python m pip install upgrade pip RUN pip install r requirements txt no cache dir Install flytectl RUN curl sL <https ctl flyte org install> | bash Copy Flyte configs RUN mkdir p flyte configs COPY config flyte dev config yaml flyte configs flyte dev config yaml CMD bash ENTRYPOINT ``` Doubt the problem is here as when using pyflyte register I manage to run all workflows

Jan Fiedler · Answer

I think you are missing a `COPY root` at some point Your workflow and src code need to be in ` root` inside the container

Nuno Martins · Answer

Are you sure Don t these get unpacked in the destinationDirectory I pass in flytectl register I don t want to rebuild the docker image everytime something in the code changes

Jan Fiedler · Answer

I think yes I think it only works like this if you have the workflow aswell as all tasks in 1 python file But i also could be wrong about it

Nuno Martins · Answer

But how come then I can do pyflyte register and run the workflows Only problem I have is with pyflyte package + flytectl register

Jan Fiedler · Answer

This is indeed suspicious how does your full pyflyte register command looks like

Nuno Martins · Answer

`pyflyte register p lt PROJECT gt d lt DOMAIN gt image myimage tag src`

Victor Churikov · Answer

I don t use pyflyte flytectl for registration but I achieve the same using `FlyteRemote` class in Python using its `register script` and `source path` + `copy all` args The source path gets copied inside the container and I don t have to rebuild my image every time Works with multiple files it copies the folder recursively so you don t have to put everything in one file

Samhita Alla · Answer

Are you fast registering your workflows < Nuno Martins>

Nuno Martins · Answer

Figured it out meanwhile Yes it was the fast registering flag missing and also needed ` d root` in pyflyte package Thanks for the help

Jan Fiedler · Answer

Nice Thanks for the update