Is there way to specify a custom type transformer ...
# flytekit
r
Is there way to specify a custom type transformer for a JSON type? We'd like to use JSON over the wire instead of pickling/some other serialization format for a particular type, but wasn't able to find any examples (most of them were around types that were serialized as blobs)
e
and what would be the type of this object?
r
A dataclass (but one that isn't decorated w/
@dataclass_json
). It has a
Generated
mixin that we'd like to register a type transformer for, and each instance of
Generated
will have a `to_dict`/`from_dict` method
Friendly bump on this thread -- is there a way to register a typetransformer that maps a class to JSON?
e
@Rahul Mehta, you're saying that after https://github.com/flyteorg/flytekit/commit/b7ecdf60768bf8b03a849947d5e2cab0fccde1e3 you are not able to define a type transformer for
Generated
?
r
Ah, that’s what im trying to implement now - just not sure what to pass in to the LiteralType. Is the recommended way to just use to/from_json to write the json to a file and mimic the behavior of one of the other type transformers?
That change will enable me to register it once I figure this out though
e
Take a look at the DictTransformer. You'll notice that in the simple case we simply struct the protobuf struct. Since you're defining your own transformer, you get to choose how the literal is going to be represented (i.e. if you want you can simply dump the json to a string and load from it, without worrying about writing that string to a file.)
r
Ah that sounds exactly like what I'm trying to do, thanks @Eduardo Apolinario (eapolinario)!
105 Views