@ancient-wolf-19325 Hi when using flyte can function decorators handle pytorch classes as a return? for example im trying to return

-> tuple[pytorch class, pytrorch class]

but I get error:

Failed to convert type <class 'torch.utils.data.dataloader.DataLoader'> to type <class
'torch.utils.data.dataloader.DataLoader'>.
Error Message: ctypes objects containing pointers cannot be pickled

Flyte can handle PyTorch classes as return types, but there are some limitations and specific configurations required. According to the Flyte documentation, Flyte supports serializing and deserializing PyTorch models and tensors using custom transformers. However, certain PyTorch objects, like

torch.utils.data.dataloader.DataLoader

, may not be directly supported due to their complex internal structures. The error you're encountering suggests that the

DataLoader

object contains ctypes objects, which cannot be pickled by default. Flyte relies on pickle for serialization, and this limitation can cause issues. To work around this, you can either: 1. Serialize the

DataLoader

to a format that can be pickled (e.g., save the dataset to disk and reload it in the next task). 2. Implement a custom transformer for the

DataLoader

type. Would you like more details on how to implement a custom transformer or an example of serializing the

DataLoader

? Sources: - Flyte PyTorch Type 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.