Zachary Carrico
08/18/2022, 3:33 PMTypeTransformerFailedError
occurs when using imported variables in workflows, but if the variables are declared in the same python module as the workflow there is no error? Thank you!Shivay Lamba
08/18/2022, 3:59 PMZachary Carrico
08/18/2022, 4:01 PMFile "<redacted>/lib/python3.9/site-packages/flytekit/core/type_engine.py", line 97, in assert_type
raise TypeTransformerFailedError(f"Type of Val '{v}' is not an instance of {t}")
flytekit-1.1.0
from flytekit import task, workflow
from m1 import MyDataclass, inputs
def f(x: list[MyDataclass]) -> MyDataclass:
return x[0]
@task
def tsk(x: list[MyDataclass]) -> None:
result = f(x)
print(result)
@workflow
def wf(x: list[MyDataclass]) -> None:
tsk(x=x)
wf(x=inputs)
m1.py
from dataclasses import dataclass
from dataclasses_json import dataclass_json
@dataclass_json
@dataclass
class MyDataclass:
a: int
inputs = [MyDataclass(1)]
Shivay Lamba
08/18/2022, 4:39 PMZachary Carrico
08/18/2022, 4:41 PMSo was there a separate implementation in your previous example?iiuc, yes, the real code is different than the example code shown here
m1.py
is importing dataclasses from a third module (eg. m3.py
). If a relative import path is used it produces the TypeTransformerFailedError
. If an absolute path is used, it worksAlireza
08/18/2022, 5:03 PMimport sys
sys.path.append('d/modules/')
Then you don't have to worry about relative or absolute path.Zachary Carrico
08/18/2022, 6:14 PMAlireza
08/18/2022, 6:27 PMZachary Carrico
08/18/2022, 6:28 PMAlireza
08/18/2022, 6:30 PM@workflow
def new_wf_test():
new_dynamic_test()
from core import inputModule # <<<<---- PAY ATTENTION HOW IMPORTED
print(inputModule.varInt)
print(inputModule.varString)
return
#!/usr/bin/env bash
set -e
pyflyte -c ~/.flyte/demo-config.yaml run --service-account demo --remote core/on_examples.py new_wf_test
Zachary Carrico
08/18/2022, 6:53 PMTypeTransformerFailedError: Type of Val 'MyDataset(name='my_name', label=0)' is not an instance of <class 'types.MyDatasetSchema'>
I’m trying to understand what’s causing the error.
afaik type(MyDataset)
is MyDataset
, not MyDatasetSchema
. Would you please help me understand why it isn’t checking that the instance is of type MyDataset
? (this is for a class MyDataset
decorated with @dataclass_json and @dataclass)Yee
Zachary Carrico
08/18/2022, 7:47 PMYee
Zachary Carrico
08/18/2022, 7:47 PMYee
Zachary Carrico
08/18/2022, 7:50 PMraise TypeTransformerFailedError(f"Type of Val '{v}' is not an instance of {t}")
L97 for flytekit.core.type_engineYee
Zachary Carrico
08/18/2022, 7:59 PM