I have a specific use-case where I'm saving my obj...
# flyte-support
b
I have a specific use-case where I'm saving my objects to disk (using @dataclasses_json.to_json()). This works well except for FlyteFile - in the case of FlyteFiles it saves the
path
(/tmp/easfdasdfas) and not the
_remote_source
(gs://flyte-asdfdasfd/asdfa). Is there a Flyte way to serialize objects to json and storing the
_remote_source
for FlyteFile?
1
d
I think my might not be the best way but you can use
.remote_source
to get the value, and set your path's value in Flytefile.
so that when you serialize your FlyteFile, your value
path
will be
_remote_source
b
thanks!! my object is a bit nested though (i.e. my FlyteFiles are a couple of levels down). Is there a way to do this automatically? (Perhaps I can use Flyte's serialization method, somehow?)
d
I'm working on a PR to customize FlyteFile's serialization behaviour in Flyte
I just inherit a Serlialize Interface from mashumaro, which is a serialization API
But it is complicated to implement it
And we didn't provide users to customize
FlyteFiles's serialization method
unless you inherit FlyteFile and write your own code
compare to that
I recommend you to iterate all dataclass attributes and modify your
FltyeFile
And in flyetkit,
FlyteFile
's serialization and deserialization behaviour is different from
FlyteFile
in dataclass.
r
I had this same problem, solved it like:
Copy code
class FlyteFileSerialize(SerializationStrategy):
    """Class to serialize FlyteFile and maintain the remote path or source."""

    def serialize(self, v: FlyteFile) -> dict[str, str]:
        return {"path": str(v.remote_path or v.remote_source or v.path)}

    def deserialize(self, v: dict[str, str]) -> FlyteFile:
        return FlyteFile(v["path"])

@dataclass
class ExampleDataclass:
    class Config:
        """Config defining serialization for mashumaro."""

        serialization_strategy: ClassVar = {
            FlyteFile: FlyteFileSerialize(),
        }
d
You can checkout my PR here
It almost do the samething
You can reference the way I write it
r
nice, I like the
SerializableType
abstraction so we can extend to other non-mashumaro supported types!
cleaner than having to set configs explicitly on the dataclasses
🙏 1
d
Will merge it tonight
🙏 1
b
sorry, for the late reply, but appreciate it! In the end, I went for a slightly different solution by basically storing all longer strings in a file, instead of serializing whole objects.
❤️ 1
d
Oh this is a nice solution too!
Thank you for the feedback!