Failed to run dynamic task in flytekit 1 9 1 cc < honnix> Le Flyte #ask-the-community

Join Slack

Failed to run dynamic task in flytekit 1.9.1 cc <@...

# ask-the-community

Kevin Su

09/27/2023, 8:54 PM

Failed to run dynamic task in flytekit 1.9.1 cc @honnix Let’s discuss here. 🧵 cc @Yee

Kevin Su

09/27/2023, 8:55 PM

it works for me though.

Copy code

from flytekit import dynamic, workflow, task, conditional
from .dynamic_define import t1, t2


@dynamic()
def d1(a: int) -> int:
    return t2(a=a)


@workflow
def wf(a: int = -4) -> int:
    res1 = d1(a=a)
    res2 = t1(a=a)
    return (
        conditional("fractionss")
        .if_(a >= 1)
        .then(res2)
        .else_()
        .then(res1)
    )


if __name__ == '__main__':
    wf()

Kevin Su

09/27/2023, 8:55 PM

@honnix could I share your example here

honnix

09/27/2023, 9:26 PM

Maybe not yet. I can do some tweak to make sure there is nothing sensitive. I hope you can understand.

Kevin Su

09/27/2023, 9:42 PM

tyty. above workflow is what I ran

Jay Ganbat

09/27/2023, 9:59 PM

any error log? stacktrace?

Kevin Su

09/27/2023, 10:42 PM

Error:

Copy code

Traceback (most recent call last):
  File "/root/.venv/lib/python3.8/site-packages/flytekit/core/tracker.py", line 18, in import_module_from_file
    module = importlib.util.module_from_spec(spec)
  File "frozen importlib._bootstrap", line 553, in module_from_spec
AttributeError: 'NoneType' object has no attribute 'loader'

Jay Ganbat

09/27/2023, 11:34 PM

does any of the other tasks works, also does the decorator need to be just

@dynamic

honnix

09/28/2023, 11:30 AM

Yes the others work. I think

@dynamic

is sufficient for a dynamic workflow. There is a bit more context for the error:

Copy code

python
Traceback (most recent call last):
  File "/root/.venv/lib/python3.8/site-packages/flytekit/core/tracker.py", line 18, in import_module_from_file
    module = importlib.util.module_from_spec(spec)
  File "<frozen importlib._bootstrap>", line 553, in module_from_spec
AttributeError: 'NoneType' object has no attribute 'loader'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/.venv/lib/python3.8/site-packages/flytekit/core/base_task.py", line 610, in dispatch_execute
    native_outputs = self.execute(**native_inputs)
  File "/root/.venv/lib/python3.8/site-packages/flytekit/core/python_function_task.py", line 180, in execute
    return self.dynamic_execute(self._task_function, **kwargs)
  File "/root/.venv/lib/python3.8/site-packages/flytekit/core/python_function_task.py", line 315, in dynamic_execute
    return self.compile_into_workflow(ctx, task_function, **kwargs)
  File "/root/.venv/lib/python3.8/site-packages/flytekit/core/python_function_task.py", line 209, in compile_into_workflow
    self._create_and_cache_dynamic_workflow()
  File "/root/.venv/lib/python3.8/site-packages/flytekit/core/python_function_task.py", line 188, in _create_and_cache_dynamic_workflow
    self._wf = PythonFunctionWorkflow(self._task_function, metadata=workflow_meta, default_metadata=defaults)
  File "/root/.venv/lib/python3.8/site-packages/flytekit/core/tracker.py", line 78, in __call__
    mod_name, mod_file = InstanceTrackingMeta._find_instance_module()
  File "/root/.venv/lib/python3.8/site-packages/flytekit/core/tracker.py", line 69, in _find_instance_module
    mod = InstanceTrackingMeta._get_module_from_main(frame.f_globals)
  File "/root/.venv/lib/python3.8/site-packages/flytekit/core/tracker.py", line 58, in _get_module_from_main
    return import_module_from_file(module_name, file)
  File "/root/.venv/lib/python3.8/site-packages/flytekit/core/tracker.py", line 25, in import_module_from_file
    raise ModuleNotFoundError(f"Module from file {file} cannot be loaded") from exc
ModuleNotFoundError: Module from file /root/.venv/bin/pyflyte-execute cannot be loaded

So it seems for some reason, tracker.py tried to work on

/root/.venv/bin/pyflyte-execute

, which is a generated thin wrapper.

2 Views

Open in Slack

Previous Next