Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

side question from the above, a structured datasets and flyteschemas interoperable at all

basically if you have a pd.DataFrame output and another pd DF input in a subsequent task, but the first task was cached, yeah yes, the new structured dataset stuff can read it.

but they’re not classes of each other in python, so you can’t write a task that outputs a FlyteSchema and another one that takes in a StructuredDataset

<@UNR3C6Y4T>, but FlyteSchema to StructuredDataset seems plausible according to <https://github.com/flyteorg/flytesnacks/blob/master/cookbook/core/type_system/structured_dataset.py> example.

If you see <https://github.com/flyteorg/flytesnacks/blob/6997c290b9795752ae8d3a246c7ba96f69d20324/cookbook/core/type_system/structured_dataset.py#L247-L252>, FlyteSchema is convertible to StructuredDataset.