https://flyte.org logo
#ask-the-community
Title
# ask-the-community
b

Broder Peters

02/21/2023, 3:45 PM
Hello people, as
pyflyte run --remote
"cannot import tasks and workflows from other files currently" I tried going the way of "use
pyflyte register
to handle those". (quotations taking from the --help output of pyflyte run) With my current setup:
Copy code
workflow/
|- ClassTest.py
|- example.py
And example.py using ClassTest.py like the following:
Copy code
from ClassTest import TestClass
from flytekit import task, workflow

@workflow
def wf():
    test_class = TestClass()
    test_class.do_stuff()
I get
ModuleNotFoundError: No module named 'ClassTest'
. What am I missing?
k

Ketan (kumare3)

02/21/2023, 3:49 PM
Can you add an ‘_init__.py file_
In workflow package
b

Broder Peters

02/22/2023, 7:57 AM
Thanks for the hint! Somehow it still doesn't work, also local execution fails now. I will double check it before I ask again as it is obviously a basic python issue on my end
My issue was that I ran
pyflyte register workflow
(with the directory only) and not
pyflyte register workflows/example.py
pointing directly to the specific workflow file... Not sure how I should get it working like here in the docs where it only points to the path: https://docs.flyte.org/projects/cookbook/en/latest/getting_started/package_register.html#iterating-on-a-flyte-project
hmm registering worked now but the execution still runs into the modulenotfounderror...
Ah, wait. I'm getting the error
AttributeError: module 'ClassTest' has no attribute 'do_stuff'
.
k

Ketan (kumare3)

02/22/2023, 4:24 PM
Hmm, can you paste the tree of the package
b

Broder Peters

02/23/2023, 7:49 AM
So my project tree looks like this:
Copy code
(flytekit) tree 
.
├── docker_build.sh
├── Dockerfile
├── LICENSE
├── README.md
├── requirements.txt
└── workflows
    ├── ClassTest.py
    ├── example.py
    ├── __pycache__
    │   ├── aws__init__.cpython-310.pyc
    │   ├── ClassTest.cpython-310.pyc
    │   ├── example.cpython-310.pyc
    │   └── __init__.cpython-310.pyc
I can execute it locally:
Copy code
(flytekit) pyflyte run workflows/example.py wf         
stufffff
If I put an init.py file next to ClassTest.py and example.py the local execution gets
ModuleNotFoundError: No module named 'ClassTest'
Register works out like this:
Copy code
(flytekit) pyflyte register workflows/example.py
Running pyflyte register from /home/bp/src/flyte/test-project/packit with images ImageConfig(default_image=Image(name='default', fqn='<http://ghcr.io/flyteorg/flytekit|ghcr.io/flyteorg/flytekit>', tag='py3.10-1.3.1'), images=[Image(name='default', fqn='<http://ghcr.io/flyteorg/flytekit|ghcr.io/flyteorg/flytekit>', tag='py3.10-1.3.1')]) and image destination folder /root on 1 package(s) ('/home/bp/src/flyte/test-project/packit/workflows/example.py',)
Registering against flyte.data-lake.url
Detected Root /home/bp/src/flyte/test-project/packit/workflows, using this to create deployable package...
{"asctime": "2023-02-23 08:40:06,054", "name": "flytekit.cli", "levelname": "WARNING", "message": "Could not determine ignored files due to:\nb'fatal: not a git repository (or any of the parent directories): .git\\n'\nNot applying any filters"}
No output path provided, using a temporary directory at /tmp/tmpvqdhys55 instead
Computed version is gkQhvVfIwtIgYDAdD6lG-A==
Loading packages ['example'] under source root /home/bp/src/flyte/test-project/packit/workflows
Successfully serialized 3 flyte objects
[✔] Registration ClassTest.do_stuff type TASK successful with version gkQhvVfIwtIgYDAdD6lG-A==
[✔] Registration <http://example.wf|example.wf> type WORKFLOW successful with version gkQhvVfIwtIgYDAdD6lG-A==
[✔] Registration <http://example.wf|example.wf> type LAUNCH_PLAN successful with version gkQhvVfIwtIgYDAdD6lG-A==
Successfully registered 3 entities
When I run the registered workflow on my cluster I get:
AttributeError: module 'ClassTest' has no attribute 'do_stuff'
"Fixed" it. I used the method without a class now as my main point was to see the execution working from different files to have a better code structure. I'm not a python pro, that's why probably my class setup was just a mess in general. But my goal is achieved for now 🎉
76 Views