https://flyte.org logo
Join Slack
Powered by
I am using FlyteRemote to launch a workflow on the...
# flyte-support
f
I am using FlyteRemote to launch a workflow on the sandbox with a custom image. I managed to get the image to execute, but getting an 'ValueError: Empty module name'.
Here is my code:
Copy code
remote = FlyteRemote(
            config=Config.for_sandbox(),
            default_project="data-processing",
            default_domain="development",
        )
        registered_wf = remote.register_workflow(
            preprocess_wf,
            version="v5",
            serialization_settings=SerializationSettings(
                image_config=ImageConfig.auto(CONTAINER_IMAGE),
            ),
        )
        execution = remote.execute(
            registered_wf,
            inputs={
                ...
            },
            name=args.name,
        )
        print(execution)
I also have a map task
Copy code
@task(container_image=CONTAINER_IMAGE)
def my_task(
    task_index: int, config: ProcessingConfig
) -> None:
    ...
I use 
map_task
in my workflow to run this task with N copies
Here is the error I'm getting when I look at K8 logs:
Copy code
Traceback (most recent call last):
 File "/usr/local/bin/pyflyte-map-execute", line 8, in <module>
  sys.exit(map_execute_task_cmd())
       ^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
  return self.main(*args, **kwargs)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
  rv = self.invoke(ctx)
     ^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
  return ctx.invoke(self.callback, **ctx.params)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
  return __callback(*args, **kwargs)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/flytekit/bin/entrypoint.py", line 585, in map_execute_task_cmd
  _execute_map_task(
 File "/usr/local/lib/python3.11/site-packages/flytekit/exceptions/scopes.py", line 143, in f
  return outer_f(inner_f, args, kwargs)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/flytekit/exceptions/scopes.py", line 173, in system_entry_point
  return wrapped(*args, **kwargs)
      ^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/flytekit/bin/entrypoint.py", line 426, in _execute_map_task
  map_task = mtr.load_task(loader_args=resolver_args, max_concurrency=max_concurrency)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/flytekit/core/utils.py", line 309, in wrapper
  return func(*args, **kwargs)
      ^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/flytekit/core/map_task.py", line 401, in load_task
  _task_def = resolver_obj.load_task(loader_args=resolver_args)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/flytekit/core/utils.py", line 309, in wrapper
  return func(*args, **kwargs)
      ^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/site-packages/flytekit/core/python_auto_container.py", line 248, in load_task
  task_module = importlib.import_module(name=task_module) # type: ignore
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
  return _bootstrap._gcd_import(name[level:], package, level)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "<frozen importlib._bootstrap>", line 1201, in _gcd_import
 File "<frozen importlib._bootstrap>", line 1114, in _sanity_check
ValueError: Empty module name
My module structure is like this:
Copy code
<repo>
\- requirements.in/.txt
\- foo_lib [python modules]
\- orchestration
  \- workflows [depends on foo_lib]
And I'm running 
PYTHONPATH=. python orchestration/workflows/data_processing/workflow.py ... (some options here)
And my code that launches workflow via FlyteRemote is under: 
if __name__ == "__main__":
in the 
workflow.py
f
hmm the module name should be 
__main__
ideally it should be 
workflow
so are you trying to run the python script?
f
I am running the python script that contains the FlyteRemote() based registration and execution code.
I wonder if I should be using register_script instead
Same error
The stack trace I am reporting above is from kubernetes pod logs
Separated the launch code into a different module and it worked. cc: @freezing-airport-6809
f
This might be an interesting bug. I bet you can reproduce with single script too
Would love help if you have time
f
Yeah, I’ll send a single script example
💯 1
@freezing-airport-6809 Here it goes:
Copy code
from flytekit import task, workflow
from flytekit.remote import FlyteRemote

from flytekit.configuration import (
    Config,
    ImageConfig,
)

@task
def say_hello(name: str) -> str:
    return f"Hello, {name}!"


@workflow
def hello_world_wf(name: str = "world") -> str:
    res = say_hello(name=name)
    return res


if __name__ == "__main__":
    remote = FlyteRemote(
        config=Config.for_sandbox(),
        default_project="data-processing",
        default_domain="development",
    )
    registered_hello_world_wf = remote.register_script(
        hello_world_wf,
        version="v5",
        copy_all=True,
        source_path=".",
        module_name="ml_platform.workflows.hello_world.workflow",
    )
    execution = remote.execute(
        registered_hello_world_wf,
        name="Testing",
        inputs={},
    )
    print(execution)
I run this via: 
PYTHONPATH=. python ml_platform/workflows/hello_world/workflow.py
And the error is:
Copy code
[1/1] currentAttempt done. Last Error: USER::
[fdaf79741488f45e5a89-n0-0] terminated with exit code (1). Reason [Error]. Message: 
args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flytekit/bin/entrypoint.py", line 500, in execute_task_cmd
    _execute_task(
  File "/usr/local/lib/python3.11/site-packages/flytekit/exceptions/scopes.py", line 143, in f
    return outer_f(inner_f, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flytekit/exceptions/scopes.py", line 173, in system_entry_point
    return wrapped(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flytekit/bin/entrypoint.py", line 372, in _execute_task
    _task_def = resolver_obj.load_task(loader_args=resolver_args)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flytekit/core/utils.py", line 309, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flytekit/core/python_auto_container.py", line 248, in load_task
    task_module = importlib.import_module(name=task_module)  # type: ignore
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1201, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1114, in _sanity_check
ValueError: Empty module name
Not a blocker for me, as moving out the launching logic into a separate module works fine, but since you needed a reproduce in case something can be improved.
❤️ 1
15 Views
Open in Slack
PreviousNext
Powered by