Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

How to defind my own datatype to defind the typedata for each task. I want to defind something not regular such as torch.model or sth like deeplake.dataset, ...

Hi Ryuu, you can define a type transformer. Docs: <https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/extending/custom_types.html>
Example plugins: <https://github.com/flyteorg/flytekit/tree/master/plugins/flytekit-onnx-pytorch>, <https://github.com/flyteorg/flytekit/tree/master/plugins/flytekit-polars>

<@U01J90KBSU9> I want to use specific this object class Dataset,  are there any simple way to do that .

<https://github.com/activeloopai/deeplake/blob/main/deeplake/core/dataset/dataset.py#L158>

<@U01J90KBSU9> Cuz if i want to using the datatype of another lib which may be very complicate to rewrite. In your solution, I can understand that I have to rewrite then register to DataType engine.

If you want add it as a native type, then you need to add a type transformer. It shouldn't be very complex to add. Else, it usually gets automatically pickled.