Hi when using flyte can function decorators handle pytorch classes as a return? for example im trying to return

-> tuple[pytorch class, pytrorch class]

but I get error:

Failed to convert type <class 'torch.utils.data.dataloader.DataLoader'> to type <class
'torch.utils.data.dataloader.DataLoader'>.
Error Message: ctypes objects containing pointers cannot be pickled.

Hey @gentle-scientist-22504, I just posted the question on #C06H1SFA19R and the answer pretty much sums up the current behavior: https://flyte-org.slack.com/archives/C06H1SFA19R/p1726676768973149?thread_ts=1726676767.636249&cid=C06H1SFA19R

so @gentle-scientist-22504 this is an excellent question. So Flyte has to move data between tasks, flytekit provides a convenient way to handle any

type

of data. The

type

is used to invoke the right

TypeTransformer

. Incase Flytekit does not find a pre-registered transformer for a type, it resolves to using pickle. Your error says - that tye type DataLoader cannot be pickled. and this was because there is no inbuilt transformer. Solution you can implement a transformer for yourself. Reason to implement a custom one, is dataloaders can be completely data dependent, distributed etc. Hard to implement a common transformer. Example for

nn.module

https://github.com/flyteorg/flytekit/blob/fb55841f8660b2a31e99381dd06e42f8cd22758e/flytekit/extras/pytorch/native.py#L31

thanks @high-park-82026 and @freezing-airport-6809 both answers were helpful