Justin Boutwell10/13/2023, 2:31 AM
pyflyte --pkgs mypackage package
that work just fine, but ran into one today that always errors when run with an error
flytectl register files
. After reading though others issue with package + register I tried adding the
ModuleNotFoundError: No module named 'mypackage'
flag, and voila, it works. My question is, why? Clearly I have some subtle difference between this and my other workflows, but what is
doing differently? The CLI help docs mention
Note this needs additional configuration, refer to the docs.
but I'm not sure what docs its referring to. It seems the difference is it may be including more of the source code than without fast. Should I just always use fast, even for those packages that clearly work without?
Lee Ning Jie Leon10/13/2023, 2:52 AM
and you import
. You would need
as you pythonpath in dockerfile (usually
only) when we fast-register, we noticed it copies the package to root as
, which is probably why it suddenly worked. So the way to work with both register is to set
Justin Boutwell10/13/2023, 3:11 AM
<root>/ mypackage/ myworkflow.py __init__.py
is done within <root> the frustrating part is this identical to others that currently work in terms of structure, so some other difference must exist and as with all these kinds of issue it does not occur with
, but I believe that is all fast by default?
Justin Boutwell10/13/2023, 1:51 PM
and I think its just an understanding issue. Below is added when run with fast to the running container:
Which must be taking the source code packaged by fast and adding it to the base image. The reason this workflow didn't work and our others did is we use
--additional-distribution s3://<bucket>/<project>/<domain>/7BPRMBPU3RS26QKXJIKS74XBBI======/fastc06f194bab8765ca864871b6ab6504ad.tar.gz --dest-dir /root
primarily, but there are a few simple cases we don't need any extra requirements. In those cases, we just use task/workflow decorations without any image and rely on Flyte's base image. Because the development flow (pyflyte run) is fast, it just works, since the source is added to the container at runtime. The production examples don't include using fast as there must be an assumption you are pre-baking all the images with the source code (in our case envd is doing this, but could just be docker build). So
really means "no pre-built container". In these simple cases pre-building a container with the source seems unecessary. I think I'll add a mechanism to conditionally package with
in these scenarios in our pipeline.