https://flyte.org logo
#ask-the-community
Title
# ask-the-community
d

Derek Yu

05/22/2023, 4:11 AM
Hi I was wondering if anyone has run into
ValueError: Empty module name
when running a
map_task()
with flytekit and trying to load the actual
task
object. More details below ⬇️ cc: @Heidi Hurst
When calling
map_task
with flytekit we are seeing this
ValueError: Empty module name
Copy code
│ ❱ 546 │   _execute_map_task(                                                 │
│                                                                              │
│ /root/micromamba/envs/xxx/lib/python3.10/site-packages/flytekit/e │
│ xceptions/scopes.py:160 in system_entry_point                                │
│                                                                              │
│ ❱ 160 │   │   │   │   return wrapped(*args, **kwargs)                        │
│                                                                              │
│ /root/micromamba/envs/xxx/lib/python3.10/site-packages/flytekit/b │
│ in/entrypoint.py:394 in _execute_map_task                                    │
│                                                                              │
│ ❱ 394 │   │   map_task = mtr.load_task(loader_args=resolver_args, max_concur │
│                                                                              │
│ /root/micromamba/envs/xxx/lib/python3.10/site-packages/flytekit/c │
│ ore/utils.py:295 in wrapper                                                  │
│                                                                              │
│ ❱ 295 │   │   │   │   return func(*args, **kwargs)                           │
│                                                                              │
│ /root/micromamba/envs/xxx/lib/python3.10/site-packages/flytekit/c │
│ ore/map_task.py:368 in load_task                                             │
│                                                                              │
│ ❱ 368 │   │   resolver_obj = load_object_from_module(resolver)               │
│                                                                              │
│ /root/micromamba/envs/xxx/lib/python3.10/site-packages/flytekit/t │
│ ools/module_loader.py:43 in load_object_from_module                          │
│                                                                              │
│ ❱ 43 │   class_obj_mod = importlib.import_module(".".join(class_obj_mod))    │
│                                                                              │
│ /root/micromamba/envs/xxx/lib/python3.10/importlib/__init__.py:12 │
│ 6 in import_module                                                           │
│                                                                              │
│ ❱ 126 │   return _bootstrap._gcd_import(name[level:], package, level)        │
│ in _gcd_import:1047                                                          │
│ in _sanity_check:981                                                         │
╰──────────────────────────────────────────────────────────────────────────────╯
ValueError: Empty module name
y

Yee

05/22/2023, 4:26 AM
that feels like the underlying task is not accessible.
where do you have that defined?
can you copy paste more code? is this fast-register?
d

Derek Yu

05/22/2023, 4:41 AM
The workflows are registered by running
flytectl register files
map_task
is invoked as such:
Copy code
result = map_task(foo_task, concurrency=5)(
        a = x 
    )
in a workflow defined in directory
flyte/a/b/tasks
foo_task
is defined in a file in the same directory
flyte/a/b/tasks
The failing command
pyflyte-map-execute
in the logs has
--input
'task-module'
:
flyte.a.b.tasks
Here is the structure of the error message:
Copy code
CalledProcessError: Command '['pyflyte-map-execute', '--inputs',
'xxx', '--output-prefix',
'xxx', '--raw-output-data-prefix',
'xxx',
'--checkpoint-path',
'xxx',
'--prev-checkpoint', '""', '--dynamic-addl-distro',
'xxx', '--dynamic-dest-dir', '.',
'--resolver', 'MapTaskResolver', '--', 'vars', 'resolver',
'flytekit.core.python_auto_container.default_task_resolver', 'task-module',
'flyte.a.b.tasks', 'task-name', 'foo_task']'
Thanks Yee, let me know if there's anything else I can provide that would be helpful to troubleshoot!
s

Samhita Alla

05/22/2023, 5:26 AM
I think the task module should be
flyte.a.b.tasks.<module>
. Is it possible for you to share the directory structure and the serialization & registration commands you're running?
d

Derek Yu

05/22/2023, 6:50 AM
Thanks Samhita, here's the dir structure: workflow defined in
flyte/a/b/tasks/workflows.py
and the workflow calls a map_task():
Copy code
@workflow
def foo_workflow(): 
    result = map_task(foo_task, concurrency=5)(
        a = x 
    )
task is defined in
flyte/a/b/tasks/tasks.py
Serialization/registration commands:
Copy code
pyflyte -k a -k flyte.a serialize --image xxx workflows -f /tmp/workflows \
	&& flytectl --admin.endpoint dns:///xxx register files /tmp/workflows/* -p foo -d bar --version='xxx' \
h

Heidi Hurst

05/22/2023, 10:20 AM
FYI dropping back down to Flytekit 1.4.2 resolved this issue, so I suspect this is something that's either broken or changed with the recent releases (the above error was encountered with flytekit 1.6.1).
s

Samhita Alla

05/23/2023, 5:04 AM
cc @Eduardo Apolinario (eapolinario)
e

Eduardo Apolinario (eapolinario)

05/23/2023, 9:59 PM
@Derek Yu, @Heidi Hurst, I'm having trouble reproing this. Can you give us more details? For example, can you show the dir structure (by running
tree
at the root of the repo)? Also, what's the type of
x
in
foo_workflow
?
h

Heidi Hurst

05/24/2023, 5:54 PM
Sure! Thanks for following up @Eduardo Apolinario (eapolinario) 🙂 I've been able to construct a small example which shows the same error: Files:
flyte/a/b/workflows.py
Copy code
@workflow
def repro_issue_workflow() -> List[int]:
    foo = foo_task()
    bar = map_task(bar_task, concurrency=3)(number=foo)
    return bar
flyte/a/b/tasks.py
Copy code
@task
def foo_task() -> List[int]:
    return [1, 2, 3]

@task
def bar_task(number: int) -> int:
    return number + 1
In this example, the type of data in the map task is
List[int]
. I've confirmed that this fails with both regular registration and fast registration. This also seems to fail when both the tasks and workflows are in the same file 🤔 In the screenshot (included) you can also see that the map task has a different naming convention than I would have expected from past flytekit versions as well, although that might be a red herring. Happy to open an issue or file a bug report if that would be helpful 🙏
87 Views