do you have any examples of class-based tasks/work...
# flyte-support
m
do you have any examples of class-based tasks/workflows instead of functions? or any examples of huge configs (like 100 options) with overload? usually we do it with hydra, but I wonder if there are any other options another question: how is the data passed between tasks, if I have huge dataset to supply for model? (in luigi we usually pass filepaths even if the file is in s3, any best practices for flyte?)
f
Hi @miniature-zebra-33903 firstly welcome to the community. Large data is automatically passed using references to a blob or underlying store that fsspec supports. This should explain how data is handled https://docs.flyte.org/en/latest/concepts/data_management.html We will share a class based example - this is the issue that folks have asked - please explain your usecase as well https://github.com/flyteorg/flyte/issues/3492
👍 1
Also you can use dataclasses to pass large data objects or any other objects
🫡 1
159 Views