Laura Lin
09/08/2023, 6:11 PM╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ <venv>/lib/python3.10/site-packages/flytekit/core/tracker.py:18 in import_module_from_file │
│ │
│ ❱ 18 │ │ module = importlib.util.module_from_spec(spec) │
│ in module_from_spec:568 │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
AttributeError: 'NoneType' object has no attribute 'loader'
The above exception was the direct cause of the following exception:
╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ <venv>/lib/python3.10/site-packages/flytekit/exceptions/scopes.py:199 in user_entry_point │
│ │
│ ❱ 199 │ │ │ │ return wrapped(*args, **kwargs) │
│ │
│ <workflow>.py:105 in workflow_name │
│ │
│ ❱ 105 │ map_task(task_name, concurrency=10)(info=[]) │
│ │
│ <venv>/lib/python3.10/site-packages/flytekit/core/map_task.py:338 in map_task │
│ │
│ ❱ 338 │ return MapPythonTask(task_function, concurrency=concurrency, min_success_ratio=min_s │
│ │
│ <venv>/lib/python3.10/site-packages/flytekit/core/tracker.py:78 in __call__ │
│ │
│ ❱ 78 │ │ mod_name, mod_file = InstanceTrackingMeta._find_instance_module() │
│ │
│ <venv>/lib/python3.10/site-packages/flytekit/core/tracker.py:69 in _find_instance_module │
│ │
│ ❱ 69 │ │ │ │ mod = InstanceTrackingMeta._get_module_from_main(frame.f_globals) │
│ │
│ <venv>/lib/python3.10/site-packages/flytekit/core/tracker.py:58 in _get_module_from_main │
│ │
│ ❱ 58 │ │ return import_module_from_file(module_name, file) │
│ │
│ <venv>/lib/python3.10/site-packages/flytekit/core/tracker.py:25 in import_module_from_file │
│ │
│ ❱ 25 │ │ raise ModuleNotFoundError(f"Module from file {file} cannot be loaded") from exc │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ModuleNotFoundError: Module from file <venv>/bin/pyflyte cannot be loaded
Kevin Su
09/08/2023, 6:22 PMLaura Lin
09/08/2023, 6:24 PM@dataclass_json
@dataclass
class Info:
argx: str
@task(
requests=Resources(mem="8Gi", ephemeral_storage="16Gi", cpu="4"),
retries=0,
)
def task_y(info: Optional[Info]) -> None:
pass
@workflow
def workflow_r():
map_task(task_y)(info=[])
from flytekit import map_task, task, workflow
@task
def task_y(info: str) -> str:
return info
@workflow
def workflow_r(inp: List[str]):
map_task(task_y)(info=inp)
@task
def my_mappable_task(a: int) -> Optional[str]:
return str(a)
@dynamic
def my_dynamic(x: List[int]):
return map_task(my_mappable_task, concurrency=10, min_success_ratio=0.75,)(
a=x
).with_overrides(cpu="10M")
@workflow
def my_wf2(x: List[int]):
my_dynamic(x=x)
@task
def my_mappable_task(a: int) -> typing.Optional[str]:
return str(a)
@workflow
def my_wf(x: typing.List[int]) -> typing.List[typing.Optional[str]]:
return map_task(my_mappable_task, metadata=TaskMetadata(retries=1), concurrency=10, min_success_ratio=0.75,)(
a=x
).with_overrides(cpu="10M")
Kevin Su
09/08/2023, 8:21 PM@task
def my_mappable_task(a: int) -> typing.Optional[str]:
return str(a)
@workflow
def my_wf(x: typing.List[int]) -> typing.List[typing.Optional[str]]:
return map_task(my_mappable_task, metadata=TaskMetadata(retries=1), concurrency=10, min_success_ratio=0.75,)(
a=x
).with_overrides(cpu="10M")
This works for me. how did you register the workflow? pyflyte run
. are you able to run this workflow locally?Laura Lin
09/08/2023, 8:22 PMYee
Kevin Su
09/08/2023, 8:22 PMLaura Lin
09/08/2023, 8:23 PMpyflyte --verbose --pkgs foldername package --fast -p flytetester -d /foldername/src --image image -o $(git rev-parse HEAD).tgz --deref-symlinks -f
pyflyte register -p flytetester --domain development --destination-dir /foldername/src src/foldername -v ${TAG} --image image
pyflyte run src/folder/test_wf.py my_wf --x="[0,1]"
I get the same errorKevin Su
09/08/2023, 8:34 PMLaura Lin
09/08/2023, 10:23 PM