New RunLLM response generated: >>>*# Question* I have a specific use-case where I'm saving my objects to disk (using @dataclasses_json.to_json()). This works well except for FlyteFile - in the case of FlyteFiles it saves the

path

(/tmp/easfdasdfas) and not the

_remote_source

(gs://flyte-asdfdasfd/asdfa). Is there a Flyte way to serialize objects to json and storing the

_remote_source

for FlyteFile? # Answer Flyte's

FlyteFile

objects can be serialized to JSON while preserving the

_remote_source

attribute by implementing custom serialization and deserialization methods. This ensures that the

remote_source

is stored instead of the local

path

. Here is an example of how to achieve this:

from dataclasses import dataclass
from dataclasses_json import DataClassJsonMixin
from flytekit.types.file import FlyteFile

@dataclass
class MyDataClass(DataClassJsonMixin):
    my_file: FlyteFile

    def to_json(self):
        data = self.__dict__.copy()
        # Replace the path with remote_source for FlyteFile
        if isinstance(self.my_file, FlyteFile):
            data['my_file'] = self.my_file.remote_source
        return json.dumps(data)

    @classmethod
    def from_json(cls, json_data):
        data = json.loads(json_data)
        # Replace remote_source with FlyteFile object
        if 'my_file' in data:
            data['my_file'] = FlyteFile(data['my_file'])
        return cls(**data)

# Example usage
my_data = MyDataClass(my_file=FlyteFile("gs://flyte-asdfdasfd/asdfa"))
json_data = my_data.to_json()
restored_data = MyDataClass.from_json(json_data)

This custom serialization ensures that the

remote_source

is stored in the JSON representation, and the custom deserialization restores the

FlyteFile

object correctly. Would you like more details on this approach? Sources: - flytekit/core/type_engine.py - flytekit/unit/core/test_type_engine.py Slack Link https://flyte-org.slack.com/archives/CP2HDHKE1/p1720767624.353849 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.