https://flyte.org
Join Slack
I am not able to find flyte documentation for the resolution of this issue. Could anyone share the r...
# flyte-support
s
I am not able to find flyte documentation for the resolution of this issue. Could anyone share the root cause of the error and the fix?
b
hi @salmon-refrigerator-32115 can you share what you’re running to get the error? As @glamorous-carpet-83516 mentioned you can use the pyflyte register command to package and register your workflow to your Flyte cluster
Copy code
pyflyte register --image <image>:<version> <workflows_directory>
Where 
<workflows_directory>
is the root directory containing all of the code you want to ship with your image
s
Sure, @broad-monitor-993 The command is:
Copy code
pyflyte run --remote refer_subworkflow_in_lib.py parent_wf --a 9
@broad-monitor-993, the folder structure is like this
In refer_subworkflow_in_lib.py file:
Copy code
from lib.subwf import op, t1, my_subwf
I run command (last param flyte is the folder name):
Copy code
pyflyte register --image dai/mlp/flyte/demo_image:1 flyte
And I got:
Copy code
from lib.subwf import op, t1, my_subwf
ModuleNotFoundError: No module named 'lib'
g
Hi @salmon-refrigerator-32115 could you try
Copy code
pyflyte register --image dai/mlp/flyte/demo_image:1 refer_subworkflow_in_lib.py
s
Hi @glamorous-carpet-83516 I tried and got the same error:
Copy code
from lib.subwf import op, t1, my_subwf
ModuleNotFoundError: No module named 'lib'
g
It’s wired, mind sharing your entire folder to me (upload to github or somewhere). I can dig into it.
s
It’s not having a git repo yet. Will that be the issue?
Also I have flyte installed in the venv in the env folder.
g
it shouldn’t need to be git repo
s
and I have conda activated
I can try deactivate conda.
@glamorous-carpet-83516, I found the reason. I have init file in the root folder. I don’t know why flyte doesn’t like it though. Do you?
After registering, I run the command below and it failed for ModuleNotFoundError: No module named ‘lib’ again in the pod.
Copy code
pyflyte run --remote refer_subworkflow_in_lib.py parent_wf --a 6
Do I need to restart the demo cluster in docker first?
Restarting demo cluster didn’t help.
g
you have to use pyflyte register, and manually trigger the workflow on flyte console
Copy code
pyflyte register refer_subworkflow_in_lib.py
t
@salmon-refrigerator-32115 the figuring out of where the base directory is is actually not trivial. the tools try to rely on the existence of init files to figure out where the base folder is.
of the given files, it walks up the folder structure until it finds a folder that doesn’t have an init file.
that folder is assumed to be the root. this doesn’t play nice with users who have namespace packages though (which is something we’re working on finding a ux-friendly way to address)
having init files set at the right place should get things to work
pyflyte run is different story - that’s only works if all your code is in a single file.
it was written as a quick and easy way to get started or iterate on things.
eventually as your repo grows you’ll want more than one file and you’ll have to move to the register/package patterns.
s
Thank you @thankful-minister-83577 for the explanations!
167 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by