Channels
announcements
ask-the-community
auth
conference-talks
contribute
databricks-integration
datahub-flyte
deployment
ecosystem-unionml
engineeringlabs
events
feature-discussions
flyte-bazel
flyte-build
flyte-console
flyte-deployment
flyte-documentation
flyte-github
flyte-on-gcp
flyte-ui-ux
flytekit
flytekit-java
flytelab
great-content
hacktoberfest-2022
helsing-flyte
in-flyte-conversations
integrations
introductions
jobs
konan-integration
linkedin-flyte
random
ray-integration
ray-on-flyte
release
scipy-2022-sprint
show-and-tell
sig-large-models
torch-elastic
workflow-building-ui-proj
writing-w-sfloris
Title
z
Zachary Carrico
08/18/2022, 3:33 PMHi, anyone have a guess as to why
TypeTransformerFailedErrors
Shivay Lamba
08/18/2022, 3:59 PMHi Zachary, thanks for your question. Do you also receive any other error message alongside the TypeTransformerFailedError
z
Zachary Carrico
08/18/2022, 4:01 PMNope. I’m avoiding copying/pasting the stack trace for company privacy reasons
I can try and reproduce it in a dummy format if that will help. More details are that the types failing when imported are annotated with @dataclass and @dataclass_json
pseudocode is this:
@task
tsk(x: list[MyDataclass]) -> None:
result = f(x)
print(result)
@workflow
wf(x: list[MyDataclass]) -> None:
tsk(x)
if the input for x is defined in the same module as the task/workflow code there’s no error, but when importing identical code (copy/pasted) from a neighbor module, it produces the error
File "<redacted>/lib/python3.9/site-packages/flytekit/core/type_engine.py", line 97, in assert_type
raise TypeTransformerFailedError(f"Type of Val '{v}' is not an instance of {t}")flytekit-1.1.0hmm, well, this works:
m2.py
from flytekit import task, workflow
from m1 import MyDataclass, inputs
def f(x: list[MyDataclass]) -> MyDataclass:
return x[0]
@task
def tsk(x: list[MyDataclass]) -> None:
result = f(x)
print(result)
@workflow
def wf(x: list[MyDataclass]) -> None:
tsk(x=x)
wf(x=inputs)from dataclasses import dataclass
from dataclasses_json import dataclass_json
@dataclass_json
@dataclass
class MyDataclass:
a: int
inputs = [MyDataclass(1)]s
Shivay Lamba
08/18/2022, 4:39 PMSo was there a separate implementation in your previous example? because when just importing class, variables it should work.
z
Zachary Carrico
08/18/2022, 4:41 PMSo was there a separate implementation in your previous example?iiuc, yes, the real code is different than the example code shown here
figured it out. In the real code the equivalent of
m1.pym3.pyTypeTransformerFailedError@Shivay Lamba, thank you for the help! Do you think this error is the appropriate error when it’s really do to the way flytekit is interpreting types based on import paths?
👍 1
a
Alireza
08/18/2022, 5:03 PMGlad you solved it. There is an alternative solution also to append the module path to sys as follows:
import sys
sys.path.append('d/modules/')🙏 1
👍 1
z
Zachary Carrico
08/18/2022, 6:14 PMthank you for this @Alireza! When I register the workflow on our Flyte cluster, the same error returns although it’s now passing locally. I’m guessing it has something to do with the import paths again, but now that they’re all absolute paths, idk what else to change. Any ideas for things to try?
a
Alireza
08/18/2022, 6:27 PM@Zachary Carrico Absolutely. One inefficient option is to dump the entire of your code (input and workflow) into one test script. Then run it. Do you think you can do that?
z
Zachary Carrico
08/18/2022, 6:28 PMty! I’m assuming that would work, but what would be the long-term solution?
a
Alireza
08/18/2022, 6:30 PMIf you give that a try. My solution is to use "relative path". This is a very standard option. I attach it here for you to look.
@workflow
def new_wf_test():
new_dynamic_test()
from core import inputModule # <<<<---- PAY ATTENTION HOW IMPORTED
print(inputModule.varInt)
print(inputModule.varString)
returnAdditional in your case I strongly recommend to use bash script like the following
#!/usr/bin/env bash
set -e
pyflyte -c ~/.flyte/demo-config.yaml run --service-account demo --remote core/on_examples.py new_wf_testz
Zachary Carrico
08/18/2022, 6:53 PMty! for reference, the full error message is
TypeTransformerFailedError: Type of Val 'MyDataset(name='my_name', label=0)' is not an instance of <class 'types.MyDatasetSchema'>type(MyDataset)MyDatasetMyDatasetSchemaMyDatasetclass MyDatasety
Yee
08/18/2022, 7:44 PMsorry, still skimming, what is MyDatasetSchema
when a workflow runs locally, flytekit will go through the same serialization/deserialization steps as in production btw.
in either case, when this serde process runs, the class that you get back is not the same.
it’s a dataclass(json) class that looks the same, but it’s not the same
z
Zachary Carrico
08/18/2022, 7:47 PMty for explaining! Do you think it’s comparing the dataclass type to the dataclass’s json type and saying they’re not the same?
y
Yee
08/18/2022, 7:47 PMyou can see that code here
i’m confused where the string “Schema” is coming from…
z
Zachary Carrico
08/18/2022, 7:47 PMauto-generated in Flyte’s error msg
y
Yee
08/18/2022, 7:48 PMi don’t think this is something added by marshmallow
maybe it is
z
Zachary Carrico
08/18/2022, 7:50 PMraise TypeTransformerFailedError(f"Type of Val '{v}' is not an instance of {t}")I think it’s doing something like
v = type(MyDataset)
t = type(MyDataset.to_json)
I’m still pretty sure it has something to do with import paths. Locally, if relative paths were used it would produce this error, but once absolute paths were used, the error went away locally, but not in the Flyte cluster
y
Yee
08/18/2022, 7:56 PMsorry do you have a mvp repro? or did i miss that above?
z
Zachary Carrico
08/18/2022, 7:59 PMnope. It’s going to be easier for me to convert the dataclasses to dictionaries for workflow input and then convert them back again into dataclasses in the workflow than it will be to create a reproducible example of this. So I will do that, but appreciate everyone looking into this