sticky-angle-28419
03/16/2023, 5:33 PMpyflyte package
to work without causing module import issues - any help would be greatly appreciated. The project is in /project
(root dir) and inside is /wf
dir with wf_10.py
where the workflow code lives, where I import tasks from main.py
in the same dir. I’m running pyflyte from /project
dir with this command pyflyte --pkgs wf packages ...
and the packaging fails with:
Loading packages ['wf'] under source root /project
Failed with Unknown Exception <class 'ModuleNotFoundError'> Reason: No module named 'main'
No module named 'main'
glamorous-carpet-83516
03/16/2023, 6:26 PM__init__.py
in /project
?sticky-angle-28419
03/16/2023, 6:26 PMproject
sticky-angle-28419
03/16/2023, 6:26 PMglamorous-carpet-83516
03/16/2023, 6:27 PMglamorous-carpet-83516
03/16/2023, 6:27 PMsticky-angle-28419
03/16/2023, 6:28 PM/project
directly and run pyflyte package
from there? What should I put as --pkgs
option value?glamorous-carpet-83516
03/16/2023, 6:32 PMsticky-angle-28419
03/16/2023, 6:33 PMsticky-angle-28419
03/16/2023, 6:36 PM__init__.py
to /project
folder but same error:sticky-angle-28419
03/16/2023, 6:36 PMLoading packages ['wf'] under source root /project
No module named 'main'
Failed with Unknown Exception <class 'ModuleNotFoundError'> Reason: No module named 'main'
sticky-angle-28419
03/16/2023, 6:37 PM--pkgs
expect absolute path to the package? I assume you can specify relative path from cwd no?glamorous-carpet-83516
03/16/2023, 6:43 PMsticky-angle-28419
03/16/2023, 6:44 PM--pkgs .
for running wf_10.py
from project root (where I ran pyflyte
command), but that doesn’t work. --pkgs ''
doesn’t work eithersticky-angle-28419
03/16/2023, 6:45 PM__init__.py
in the /project
and then running pyflyte --pkgs wf package ...
there with the code inside /wf
didn’t work eitherglamorous-carpet-83516
03/16/2023, 6:56 PMglamorous-carpet-83516
03/16/2023, 6:57 PMglamorous-carpet-83516
03/16/2023, 6:58 PMsticky-angle-28419
03/16/2023, 7:09 PMsticky-angle-28419
03/16/2023, 7:10 PMpyflyte --pkgs wf package …
?sticky-angle-28419
03/16/2023, 7:11 PMmain.py
in my workflow file wf.py
- and the error is main
module does not existsticky-angle-28419
03/16/2023, 7:12 PMfrom main import task1
in wf.py
is not working because pyflyte
was run in /project
- so I think it’s expecting my from main
to be from wf.main
, but I’d like to just use main
as the modulesticky-angle-28419
03/16/2023, 7:12 PMsticky-angle-28419
03/16/2023, 7:14 PMwf.py
is the problem I’m havingsticky-angle-28419
03/16/2023, 7:14 PM--pkgs
sets the path in PYTHON_PATH, but it doesn’t seem to?glamorous-carpet-83516
03/16/2023, 8:40 PM--source ./project/wf
in the command,glamorous-carpet-83516
03/16/2023, 8:40 PMsticky-angle-28419
03/16/2023, 8:58 PMsticky-angle-28419
03/16/2023, 8:58 PMsticky-angle-28419
03/16/2023, 9:19 PM/project
, I tried pyflyte --pkgs wf package --source ./wf
and using it now gives an error No module named 'wf'
instead of No module named 'main'
- so it looks like --source
sets the project folder, but by doing that, --pkgs
is now breakingsticky-angle-28419
03/16/2023, 9:20 PM--pkgs
in this case though since neither the current dir .
or empty ''
workssticky-angle-28419
03/16/2023, 9:20 PMsticky-angle-28419
03/16/2023, 11:55 PM/project/wf
whereas main.py is in /project
. That way I could run pyflyte
from /project
and then set --pkgs
to wf
. This seems like a bug to me. I should be able to specify the cwd as --pkgs
(e.g. --pkgs .
)