There are several good comparison articles between Flyte and Airflow, but I haven't seen many comparisons to other orchestrators. I'm particularly interested in Dagster. Any plans to add something like this to a FAQ or blog post?

We have one comparing Flyte with Kubeflow: https://www.union.ai/blog-post/production-grade-ml-pipelines-flyte-vs-kubeflow. We definitely can put together a doc to list out the differences between Flyte and Dagster but that might take some time. @shy-grass-43338 @freezing-airport-6809, what do you think?

Airflow and Kubeflkow are quite different to Flyte

@best-glass-60363 there are a lot of differences with Dagster. We are more vertically integrated compute

In term of low level user API (task, workflow, op), I think it's quite similar. But dagster recently added higher level api that they call software defined asset. It's more opinionated but some might find it useful, especially for ETL. I think it's also doable in Flyte. I agree with @freezing-airport-6809, the key differentiator of Flyte is its scalability and plugability. It's hard to achive multitenancy in dagster (at least from what I experience a while back) and plugin can only be done in front end as python task. In flyte there is backend plugin concept.

Gotcha, that is a helpful differentiation. I does seem that Dagster is generally much more Python centric whereas Flyte tries to be language agnostic.

Oh yeah that part feels clunky to me. Is it still like that? All users code are stored in a deployment instead of persistent storage. And to have different set of dependency then you'll need separate deployment.